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Executive Summary 


In the forefront of serving children, youth, and their families in under-resourced New York City 
communities, Good Shepherd Services (GSS) has developed a school model for young people 
who have not been successful in traditional high schools. Built on a program model that was 
initiated at the request of the Department of Education in 1980 for neighborhood youth off 
track to graduation, it became an independent diploma-awarding high school in 2002. In 
collaboration with the New York City Department of Education (NYC DOB), transfer schools 
serve students who have fallen behind and are unlikely to graduate from high school before they 
turn 21. 


Over-age students accounted for more than two-thirds of the dropouts of the city’s Class of 
2012, but only about one in five graduates of that class were ovet-age (NYC DOE, n.d.-a). 
Youth without a diploma face long odds for economic success; over the course of a lifetime, 
high school dropouts are estimated to earn $400,000 less than high school graduates (The Annie 
E. Casey Foundation, 2012). The transfer schools are targeted at this population of over-age and 
under-credited students and are designed to provide more support in an alternative model of 
schooling. 


Taking a holistic approach, the GSS transfer schools offer a full-day, year-round academic 
program that integrates intensive support services and youth development practices with 
personalized, standards-based instruction. The schools operate under an equal partnership of 
GSS and NYC DOE leaders. 


GSS engaged Metis Associates to evaluate the effectiveness of their transfer school model in 
improving student outcomes. The study seeks to contribute to the knowledge base about 
strategies for helping disconnected youth re-connect with school by utilizing a rigorous impact 
evaluation design, as well as describing the GSS model and the population served. Conducted 
from 2011-2014, the study examines the GSS model implemented at South Brooklyn 
Community High School, New York City’s first independent transfer school (established in 
2002) and at West Brooklyn Community High School, where the model was replicated in 2006. 
While a lot has been written about best practices for serving youth, the evidence from rigorous 
studies remains limited. Furthering our understanding of what works is especially important in 
light of national conversations about improving educational and life outcomes for students who 
have become disengaged. 
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The evaluation addresses the following research questions: 


e To what extent is the GSS transfer school model implemented as designed, and how does 
the model differ from other schools? 


e What impact does the model have on student academic performance, including school 
attendance, credit attainment, persistence in school, and graduation? 


e What are the youth development outcomes of GSS transfer school students and what is the 
relationship between these outcomes and academic performance and other characteristics? 


The evaluation used a rigorous quasi-experimental design, in which outcomes for GSS students 
are compared to outcomes for equivalent comparison groups based on propensity score 
matching. For this study, comparison students were drawn from students enrolled in other 
Brooklyn transfer schools not affiliated with GSS. These schools have a similar core educational 
program but may vary in practices such as admissions criteria and partnership roles and 
responsibilities. This type of design is expected to allow for the difference in observed outcomes 
to be confidently attributed to the treatment (i.e., GSS transfer school model), meeting evidence 
standards according to the What Works Clearinghouse. Impact analyses were conducted on data 
available through the end of the 2012-13 school year, with additional analyses on graduation and 
persistence conducted through the end of 2013-14. 


Information about implementation at the GSS schools was obtained during the 2011-12 and 
2012-13 school years through surveys and focus groups of staff and students, interviews with 
school leaders, and a review of documentation. Implementation was explored in-depth in two 
previous reports; the findings are summarized herein. Information about the schools attended by 
comparison group students was obtained through interviews with representatives of the 
community organizations serving these schools as well as public information. 


In addition to examining academic outcomes, the evaluation included a measure of youth 
development of the GSS students, namely the Search Institute’s Developmental Assets Profile 
(DAP). The qualities measured by the DAP have been identified as being essential to healthy 
adolescent development and include positive relationships, opportunities, skills, values, and 
protection against risk behaviors. This aspect of the study was designed to further understanding 
about the presence of these qualities that have been shown to predict a variety of outcomes for 
youth, including academic achievement. 


Implementation Findings 


The GSS transfer school model includes programmatic approaches that have been demonstrated 
to reduce dropout rates, including assigning adult advocates to students at risk of dropping out, 
providing academic support and enrichment, personalizing the learning environment and 
instruction, providing rigorous and relevant instruction to engage students in learning and 
provide the skills needed to graduate and later in life (Dynarski et al, 2008). The schools are 
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neighborhood-based and led by a NYC DOE principal and a GSS director in equal partnership, 
with each responsible for different aspects of the school. 


Just over half of the GSS students (54%) are male. The predominant ethnic group is Hispanic 
(67%)—+eflecting the population of the schools’ neighborhoods; 15% are Black, 12% are White, 
and 6% are Asian or ‘Other’. Most students are low-income, with 75% eligible for free/reduced 
price meals. Special education students account for 9% and English language learners, 2%. 


In the year prior to enrolling in the GSS schools, the GSS students had attended school, on 
average, only 67% of the time, equivalent to missing about three months of school. On average, 
students enrolled in the two GSS schools during the study period entered the school with only 
14.8 credits despite being 17 years old.! In fact, 40% of GSS students entered with less than 11 
credits. A 17-year-old student who is on-track for graduation would be expected to have at least 
30 credits. 


The average length of time the GSS students have stayed at the schools is almost 22 months. 
During this time, their advocate counselor provides attendance outreach, social and emotional 
counseling, academic advisement and programming, work with families, and activities focused 
on the students’ postsecondary future and community building. With more than half of the 
students having the same advocate counselor since they enrolled, and most of the others having 
just two advocate counselors, the schools have attained a high degree of stability in this 
relationship. The primary person approach, as well as the size of the schools (150 to 200 
students, each) and classes (maximum 25, with most at 18 students), offers opportunities for 
individual attention; students described the relationship with staff as “familial.” 


Students are organized in ungraded cohorts and have personalized schedules that reflect their 
individual course and credit needs. Academic expectations are clear and students receive 
feedback on their performance through biweekly benchmark reports and meetings, enabling 
them to see their progress as well as where they are falling short, and the connection between 
attendance and grades—a critical aspect for students who have a history of truancy. 


A high degree of student engagement was evident in student survey responses to questions 
about participation in class as well as leadership opportunities; staff perceived the importance of 
student leadership as well. However, during the second year of the study some activities were 
curtailed because of Superstorm Sandy, which severely affected one school and its surrounding 
community. 


! A student who is on-track going into high school would be 14 years of age and expected to earn at least 10 
credits in each of four years of high school. 
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The schools also help focus students on the future, with information about postsecondary 
options, including college and careers, through workshops, seminars, postsecondary planning, 
and internships. Close to two-thirds of the students enrolled during the study period had 
participated in an internship. 


A variety of methods are used to facilitate the partnership and develop staff capacity to 
implement the youth development model; more than half of the staff rated their schools’ 
orientation efforts ‘good’ or ‘excellent;’ however further development in this area is suggested by 
the more than one-third of the staff who gave lower ratings (adequate, fair, or poor). 
Information about how the other transfer schools from which comparison students were drawn 
implement their model or program is incomplete. However, the other schools vary in the extent 
to which decision-making is shared between NYC DOE and community organization staff. An 
obvious difference is that the GSS schools are neighborhood-based. 


Academic Outcome Findings and Program Impact 


The evaluation examined the impacts of the GSS model on students’ academic and behavioral 
outcomes, including graduation, persistence (continued enrollment) in school, credits earned, 
core Regents exams passed, school attendance, and suspensions. Analyses compared the GSS 
students to comparison students overall and, when the number of students was sufficient, 
exploratory analyses were conducted for female and male students, Hispanic males, Black males, 
and students who entered the transfer school with less than 11 credits. 


Key findings for the overall groups as well as for the subgroups are presented below: 


e The GSS students have a higher probability that they will graduate from high school than the 
matched comparison students, 63% vs. 51.4%, a statistically significant finding. 


o Within the female subgroup, GSS students have a significantly higher probability 
of graduating than the comparison students, 67.8% vs. 53.2%. 


o Among those with less than 11 credits at admission, GSS students also have a 
significantly higher likelihood of graduating than the comparison students, 
52.6% vs. 35.6%. 


e Overall, the GSS students did not differ significantly from the comparison group in their 
probability of remaining in school; however the difference was in the direction of favoring 


the GSS group (30.9% vs. 26.3%). 


o For this outcome, none of the subgroup analyses achieved statistical significance 
either, although female GSS students were expected to persist in school to a 
greater extent than their comparisons. 
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e The GSS model had a statistically significant positive impact on credit attainment, measured 
by the number of credits students earned after they were admitted to the transfer schools. 
GSS students earned, on average, 4.3 more credits than the comparison group, another 
substantively importance difference. 


o The majority of subgroup analysis results for credits earned were consistent with 
the overall findings; GSS females, males, Hispanic males, and students who 
enrolled with less than 11 credits each outperformed their comparison group. 


o There was no difference in credits earned between the Black male GSS students 
and comparison group students who were Black males. 


e The GSS students were not significantly different in the average number of core Regents 
exams passed after admission when compared to students in the comparison group. 


e With a 65.4% attendance rate over the two school years, the GSS students had significantly 
better school attendance than their matched comparisons, after controlling for a number of 
student and school characteristics. The GSS students outperformed comparison students by 
5.4 percentage points, a difference that is considered substantively important by the What 
Works Clearinghouse. 


o Most of the subgroup analysis results for school attendance were consistent with 
the overall impact finding: GSS females, males, Hispanic males, and students 
with less than 11 credits at admission each outperformed comparison students. 


o The results for Black males were not statistically significant, although the 
direction of the difference favored GSS students. 


e The GSS model did not have any impact on student suspensions; however, the low number 
of suspensions for both GSS and comparison students suggests that this event has little 
relevance to this student population. 


Youth Development Findings 


The evaluation included an assessment of developmental assets as measured by the 
Developmental Assets Profile (DAP), a tool designed to assess how youth are faring personally 
and socially. DAP items are grouped into 40 assets with half categorized as external (related to 
experiences and relationships with others, and involvement in extracurricular activities) and half 
categorized as internal qualities (e.g., motivation, responsibility, decision making, and identity). 
Research suggests that youth who report more assets are more likely to report engaging in 
positive, socially constructive behaviors and less likely to report engaging in risky behaviors 
(Benson et al., 2011). A higher level of developmental assets also is associated with higher 
academic achievement (Scales et al., 2006). 


In this report, results are presented for a group of GSS students who were new to the schools in 


the fall of 2012. A longitudinal analysis of the students’ responses on the DAP was conducted 
from fall (pre) to spring of 2013 (post). The analysis assessed whether there were any changes in 
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assets over the school year and, if so, in which areas, as well as to identify any characteristics that 
predicted the students’ end-of-year scores. The study period falls within an acceptable timeframe 
for assessing change and observing the possible influence of the GSS school experience on 
students’ asset levels, nevertheless the school year may be too short a time to observe change. 
Furthermore, there was a major disruption at the beginning of this school year from Superstorm 
Sandy, which affected the lives of students and their families, and school routines, and may have 
affected students’ feelings in a variety of ways. 


The analyses conducted to identify the predictors of DAP scores confirm the relationship 
between student level of assets and academic progress (credit accumulation), the relationship 
between assets and Constructive Use of Time, and the relationship between assets and civic 
engagement. The Constructive Use of Time subscale, which includes involvement in arts and 
sports (as well as involvement in religious activities), is a domain that may be directly influenced 
by the GSS transfer school experience. The civic engagement finding is based on students who 
indicated, on the evaluation survey, that they helped to solve community problems as part of 
their GSS school experience. 


The number of assets places the youth within one of four levels of strength/risk, defined as: at- 
tisk, vulnerable, healthy, and thriving. The fall to spring analysis showed a positive trend, with an 
increase in the percentage of GSS students at the high end of assets (thriving) and a decrease in 
the percentage of students at the low end (at-risk). However, results for students in the two 
middle levels (healthy, vulnerable) were mixed. Several of the DAP subscales offer some insight 
into areas related to the GSS model. The external domains in which students improved included 
students’ feelings of empowerment (which includes having useful roles and responsibilities and 
feelings of safety) and constructive use of time (involvement in sports and arts, as well as 
religious activities). Students’ feelings of support decreased, although it should be noted that 
several of the items that make up this subscale relate to support from parents, family members, 
and neighbors. However, this subscale includes support from other adults and a school “that 
cares about kids and encourages them.” 


For the most part, students’ scores in the internal domains, comprised of items that assess 
students’ commitment to learning, positive values, personal responsibility and decision-making, 
and locus of control, decreased. Of particular interest, however, is a positive increase in the 
percentage of students who value “serving others in my community.” 


Discussion and Recommendations 


Results from the impact evaluation indicate that, compared to matched groups of over-age, 
under-credited students attending other transfer schools, the GSS transfer school model has a 
significant impact on graduation as well as the intermediate outcomes of credit attainment and 
school attendance. The probability of GSS students graduating (63%) exceeds that of the 
comparison group by 12 percentage points. The rate for GSS students also is higher than the six- 
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year graduation rate (Class of 2012) for over-age and under-credited (OA-UC) students in 
transfer schools citywide (41%) and far exceeds the rate for OA-UC students in traditional high 
schools (29%) (NYC DOE, 2014). 


The most notable findings of the subgroup analyses were that female GSS students and students 
who entered the transfer schools with less than 11 credits have a significantly higher probability 
of graduating than the comparison students (15 and 17 percentage points higher, respectively). 
Of note, students admitted with less than 11 credits likely face the most challenging road to 
graduation due to their need to complete more than three years of high school coursework 
before they turn 21. 


Other significantly positive findings on school attendance and credit attainment were found for 
the subgroups of females, males, and Hispanic males, which make up the majority of the GSS 
student population, and students with less than 11 credits at admission. 


The absence of a comparison group for the assessment of youth development limits our ability 
to assess the impact of the GSS model on changes in these areas. Nevertheless, the DAP analysis 
findings confirm the relationship between assets and academic performance, as well as 
participation in extracurricular activities and civic engagement. The results also identified some 
areas that warrant further exploration, including whether the program model is doing enough in 
terms of providing socio-emotional supports to students (and to students’ families) in order to 
bolster students’ internal assets. 


Recognizing that the final year of the implementation evaluation was a time of substantial 
disruption and curtailment of school activities because of Superstorm Sandy, the findings speak 
to the importance of consistently engaging students in learning and emphasizing the future focus 
of their education so that students are able to stay committed to continuing their education and 
see a connection to their future well-being. In working toward this goal, the schools could 
capitalize on their neighborhood base and the value that students place on civic engagement. 


Based on the study findings, we offer the following recommendations for strengthening the 
model: 

1. Engage in further professional staff development about the GSS transfer school model 
as it relates to and incorporates youth development to ensure that all staff are fully 
oriented to and comfortable with the model. 

2. Review strategies used to support students and whether there are additional ways to 
support their families through other GSS services. 

3. "To capitalize on the positive findings associated with student engagement (Le., 
Constructive Use of Time), review the extracurricular and civic engagement 
opportunities available to students with an eye toward expanding student interest in 
these areas and helping them to see the connections to academic performance and future 
well-being. 
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The following recommendations are offered for further research: 

4. Conduct an impact study with additional cohorts to build the evidence base for the 
impact of the GSS transfer school model. 

5. Conduct a qualitative study of female students and students who entered the GSS 
schools with less than 11 credits to understand better why the GSS model produced 
such strong effect with these two groups. 

6. Explore the impact of the GSS transfer school model on students’ socio-emotional 
learning and civic engagement using a comparison group evaluation design. 
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Introduction 
| 


Recognizing that traditional high schools do not work well for all students, educators and 
advocates have worked over the past decade to develop alternative settings and strategies to help 
these students stay in high school, graduate, and gain the skills they need to be successful 
(Dynarski et al, 2008). Some of these students will opt for an alternative credential to the 
traditional diploma while others will look for a program that allows them to fulfill work or family 
responsibilities during the day and continue their education in the evening. But many students 
still hope for a full high school experience that leads to a diploma. Built on a program that was 
initiated in response to the large number of dropouts in the local zoned public schools in 1980, 
Good Shepherd Services (GSS) pioneered the development of the transfer school model. In 
2002, South Brooklyn Community High School, a diploma awarding school, was established in a 
partnership with the New York City Department of Education (NYC DOE). The model was 
replicated in 2006 with a second partnership at West Brooklyn Community High School. 


The model developed by GSS takes a holistic approach to educating and supporting youth 
though an equal partnership of GSS and NYC DOE. The model offers a full-day, year-round 
academic program that integrates intensive support services and youth development practices 
with personalized, standards-based instruction. Core principles include setting high expectations 
for students, offering an active and rigorous learning environment, building healthy relationships, 
promoting student voice and responsibility, and building community. 


New York City’s on-time graduation rate was 66% in 2013 (NYC DOE, n.d.-b), below the 
national average of 80% (Stetser and Stillwell, 2014). The need for a high school model that 
supports students who are over-age and under-credited (OA-UC), the target population for 
transfer schools, is demonstrated in their large numbers and poor outcomes. A study 
commissioned by the NYC DOE estimated the number of in- and out-of-school youth who 
were at least two years behind their expected age and credit accumulation for graduation to be 
140,000 (NYC DOE, 2006). Those who remain in school face poor odds for graduation. Indeed, 
the six-year graduation rate (Class of 2012) was only 29% for over-age and under-credited 
students enrolled in traditional high schools and 41% for OA-UC students in transfer schools 
citywide (NYC DOE, 2014). Over-age students accounted for more than two-thirds of the city’s 
dropouts (Class of 2013), but only about one in five graduates of that class were over-age (NYC 
DOE, n.d.-a). Youth without a diploma face long odds for economic success. Over the course 
of a lifetime, high school dropouts are estimated to earn $400,000 less than high school 
graduates (The Annie E. Casey Foundation, 2012). 
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Thus the stakes are high for identifying a model that 
engages over-age and under-credited students and 
supports them on the path to high school graduation, 
college and careers. Developed specifically for this 
population, transfer schools are small, academically 
rigorous diploma-granting full-time high schools for 
youth between the ages of 16 and 21 who have fallen 
behind in a traditional high school. The schools offer 
an accelerated schedule so that students can earn the 
credits they need to graduate before they reach the 
age of 21. More than 10,000 students are enrolled in 
New York City’s 48 transfer schools. 


In 2011, Good Shepherd Services commissioned 
Metis Associates to evaluate the effectiveness of their 
transfer school model in improving student 
outcomes. By describing the GSS model and the 
population served, and utilizing a rigorous impact 
evaluation design, the study seeks to contribute to the 
knowledge base about strategies for helping 
disconnected youth re-connect with school. The 
evaluation examines the model as it has been 
implemented at South Brooklyn Community High 
School in the Red Hook section of Brooklyn and 
West Brooklyn Community High School in Sunset 
Park. 


The South Brooklyn Community High 
School neighborhood of Red Hook, with 
roots in the shipping industry, has some of 
the last commercial piers in New York and a 
unique mix of cobblestone streets and 
condemned buildings, housing projects, and 
the big-box store Ikea (Bagli, 2008). In the 
1960s, this part of the borough was cut off 
from neighboring areas by the Brooklyn 
Queens Expressway (BQE), solidifying the 
divide between brownstone Brooklyn and 
this industrial outpost. The neighborhood is 


On average, students enrolled in the hyo 
GSS schools during the study period had 
earned only 14.8 credits at the time of 
entry, despite being 17 years of age. In 
fact, 40% of GSS students entered with 
less than 11 credits. A 17-year-old 
student who is on-track for graduation 
would be expected to have at least 30 
credits. Thus, the average GSS student is 
about three years older than a traditional 
high school student, but has credits equal 
to only about a year and a half of school. 
In the year prior to enrolling in the GSS 
schools, the GSS students had an average 
daily school attendance of only 67%, 
equivalent to missing about three months 


of school. 


Just over half of the GSS students (54%) 
are male. The predominant ethnic group 
7s Hispanic (67%)—reflecting the 
population of the schools’ neighborhoods; 
15% are Black, 12% are White, and 
6% are Asian or ‘Other’. Most students 
are low-income, with 75% eligible for 
free/ reduced price meals. Special 
education students account for 9% and 


English language learners, 2%. 
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43% Hispanic, 36% non-Hispanic black or African American, and 17% non-Hispanic 
white (“New York City Census FactFinder,” n.d.).2 A low-lying area, Red Hook was 
severely affected by the October 2012 Superstorm Sandy. 


The West Brooklyn Community High School neighborhood of Sunset Park, situated 
along the Upper New York Bay, also was shaped by the shipping industry and the BQE, 
which separates most of the neighborhood from the waterfront. Sometimes referred to 
as “Little Latin America,” Sunset Park is 41% Hispanic. Non-Hispanic residents include 
31% Asian, 23% white, and 3% black or African American (NYC DCP, n.d.-a). Sunset 


Park’s Hispanic community includes immigrants from Mexico and Puerto Rico. 


Conducted over the two-year period from 2011 through 2013, with selected impact analyses 
continued through 2014, the study was designed to answer the following research questions: 


e To what extent is the GSS transfer school model implemented as designed, and how does 
the model differ from other schools? 


e What impact does the model have on students’ academic performance, including school 
attendance, credit attainment, persistence in school, and graduation? 


e What are the youth development outcomes of GSS transfer school students and what is the 
relationship between these outcomes and academic performance or other characteristics? 


? These data are based on statistics for census tracts 53, 59, and 85 belonging to Red Hook: Kings County 
(Brooklyn). 
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Evaluation Questions and Design 


The evaluation was designed to determine the extent to which the transfer school model is 
implemented as designed and what impacts it has on students’ academic performance and 
behavior. Three basic assumptions are being tested: 


e Students in the GSS transfer schools will demonstrate better academic outcomes than 
students in the comparison group. 

e Students in the GSS transfer schools will demonstrate better school-day attendance than 
students in the comparison group. 

e Students in the GSS transfer schools will demonstrate better behavioral outcomes than 
students in the comparison group.? 


The outcome evaluation follows a rigorous quasi-experimental design that included the 
development of equivalent comparison groups based on statistical matching. This type of design 
is expected to allow for the differences in observed outcomes to be confidently attributed to the 
treatment,* in this case, the GSS transfer school model. The design compared students at the 
GSS schools with other over-age and under-credited students enrolled in schools with a similar 
core educational program and in geographical proximity to the target schools (i.e., other transfer 
schools in Brooklyn). Thus, both their characteristics and motivations were expected to be 
similar. However, the transfer schools attended by students in the comparison group operate 
outside of the network of GSS schools and do not have a relationship with GSS.5 


The study included an assessment of implementation. The implementation evaluation was 
designed to provide an understanding of the GSS transfer school model and the extent to which 


3 As measured by number of suspensions. 


4 According to the What Works Clearinghouse (WWC) Procedures and Standards Handbook (wv 2.1, 2011), a quasi- 
experimental design based on closely matched comparison groups could meet the WWC evidence standards, 
albeit with reservations owing to the fact that unobserved variables may not be equated between the two 


groups. 
> A second group of similarly over-age and under-credited students who remained in a traditional high school 
was identified. For details about that group and why it was not comparable, see Appendix B. 
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the model’s core principles and components are evident. Information was also obtained on the 
practices in the schools from which the comparison group students were drawn. 


In addition to examining academic outcomes, the evaluation included a measure of youth 
development of GSS students. This part of the study was designed to further our understanding 
about the presence of experiences and qualities that have been shown to predict a variety of 
outcomes for youth, including academic achievement. These include positive relationships, 
opportunities, skills, and values. 


A summary of the methods used in the evaluation, and the study’s strengths and weaknesses, are 
summarized below. Details are provided in Appendix A. 


Methods Used for the Outcome Evaluation 


All of the data needed for assessing student academic and behavioral outcomes were obtained 
from school records provided by the NYC DOE. Using sophisticated propensity score matching 
(PSM) procedures, Metis matched the target students who were enrolled in the two GSS schools 
during the 2011-12 school year to similar students in the remaining 11 transfer high schools in 
Brooklyn that do not follow the GSS model. The list of baseline variables used for one-to-one 
matching included student gender, race/ethnicity, free/reduced price lunch (FRL) eligibility, 
English language learner (ELL) and special education status, New York State Grade 8 English 
Language Arts (ELA) and Math exam scores, number of years over-age when first enrolled in 
Grade 9, credits earned and average daily attendance (ADA) in Grade 9.° 


After generating a comparable non-participating group for the target sample, Metis conducted 
regression analyses for the following intended academic and behavioral outcomes: (1) graduation 
vs. discharge status, (2) enrollment vs. discharge status, (3) number of credits earned, (4) number 
of core Regents exams passed, (5) average daily attendance (ADA) during the school years 2011- 
12 and 2012-13, and (6) number of suspensions during the school years 2011-12 and 2012-13.7 
Due to the multiple dimensions of the data structure in the study (i.e., students clustered within 
schools), hierarchical linear modeling (HLM) served as the major analytic technique to account 


6 In addition, student grade level in the 2011-12 school year was used during matching (ie., a 10" grade GSS 
student would be matched to another 10‘ grader in the comparison transfer schools). See additional details in 
Appendix A. 


7 Linear regressions were used for continuous outcome measutes (i.e., credits earned, number of core Regents 
exams passed, attendance and suspensions), whereas logistic regressions were employed when outcome 
measures were dichotomous (i.e., graduation vs. discharge status, and enrollment vs. discharge status). In 
addition to the treatment indicator, all of the matching variables were included as predictors in the full 
regression models for further statistical control. 
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for the clustering effect and control for multiple covariates at each level within the same analysis 
(see model specifications in Appendix D). Standard multiple regressions were also used to re- 
analyze data when HLMs did not converge.® In addition to the overall impact analyses, 
exploratory analyses were also conducted for the following subgroups when the number of 
students was sufficient to support the analysis:° (1) males, (2) females, (3) Black males, (4) 
Hispanic males, and (5) students with less than 11 credits at admission. In addition to assessing 
intended program outcomes based on statistical significance level, Metis also calculated 
applicable effect size indices to help measure the practical importance of the findings. 


Anticipating that graduation outcomes for the transfer school population are typically longer 
term, Metis analyzed GSS and comparison student data on graduation and enrollment 
(persistence) outcomes after an additional school year. For this analysis, school records for the 
2013-14 school year were obtained from NYC DOE, and the updated graduation, enrollment, 
and discharge data were integrated into the data files created for the first round of analysis. The 
updated data files included the latest graduation vs. discharge status and enrollment vs. discharge 
status as of the end of the 2013-14 school year for the target students who were enrolled in the 
two GSS schools during the 2011-12 school year and the potential comparison students in the 
same group of transfer high schools described above. To ensure that there were no covariate 
imbalances with the addition of the new outcome data, a second round of matching procedures 
was carried out to establish the baseline equivalence of the treatment and comparison groups. For 
both updated outcomes, post-matching regression analyses (see model specifications in 
Appendix E) were conducted again for the overall matched sample and same subgroups listed 
above. 


Strength and Weakness of the Outcome Study 


The key to success for any approach to estimating the impacts of an intervention is its capability 
of projecting what student performance would have been in the absence of the intervention. 
While random assignment to treatment and control conditions would provide the strongest 
evidence of program effects, it was not feasible in the evaluation of the GSS model. As stated 
above, the GSS outcome study was based on a quasi-experimental closely matched comparison 
group design. PSM is currently considered the best available approach to generating a 
comparable group of non-participants in the absence of random assignment. Guided by the 
What Works Clearinghouse (WWC) Procedures and Standards Handbook (v 2.1, 2011), Metis ensured 


that the matching included important observed baseline characteristics related to the outcomes 


8 Metis used this procedure when the between-school variance was zero. 


° The business rules were: for continuous outcome measutes, if the N was less than 30, regression analysis 
would not be conducted (Gravetter, F. J., & Wallnau, L. B., 2013); for dichotomous outcome measures, if the 
N was less than 50, regression analysis would not be carried out (Peduzzi, P. et al., 1996). Due to low Ns of 
Black males, pertinent subgroup analyses were not performed for some outcomes. 
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of interest and eliminated the apparent selection bias based on the assessment of post-matching 
covariate balance. 


To ensure that the baseline equivalence of matching covariates could be established for each 
final analytic sample, rigorous matching was conducted multiple times with consideration given 
to the availability of pertinent outcomes for analysis. In other words, matching procedures were 
repeated whenever there were a large number of individuals missing any given outcome. This 
was done to ensure that the outcome analyses actually compared groups that were similar based 
on all selected baseline characteristics, while maximizing the number of matched pairs with both 
complete matching and outcome data. Separate PSMs resulted in matched comparison groups 
that were not necessarily constituted from exactly the same set of students, although there could 
be a substantial overlap across different matched samples (i.e., some comparison students were 
selected more than once during multiple matching). Thus, there was what could be termed a 
separate or unique analysis sample for each outcome.!? 


For each analysis sample, more than 90% of the original target sample with complete matching 
and outcome data successfully found a match (i.e., the reduction rate was less than 10%). The 
well-established baseline equivalence of the GSS group and its matched comparison group was 
capable of achieving high levels of internal validity. This means that any conclusions about a 
given outcome based on the study could be confidently attributed to the GSS model rather than 
other factors. Further, the low reduction rate provides assurance that the analytic target sample 
with matched comparisons is representative of the original treatment group with complete data 
for outcome analysis.!! 


A well-known limitation of rigorous quasi-experimental designs is the inability to account for the 
unmeasured factors (i.e., hidden selection bias) that would play a role in affecting intervention 
participation and target outcomes. In the evaluation of the GSS model, the potential comparison 
students were limited to those who were enrolled in non-GSS affiliated schools with similar core 
educational programs and geographical proximity to the target GSS schools (ie., other Brooklyn 
transfer schools) to control better for unmeasured confounding variables. The remaining 
transfer schools in Brooklyn use similar criteria for admission, and the students in these schools 
are anticipated to have similar motivation as the GSS students to meet academic requirements 
and graduate. As with any quasi-experimental design, the findings should be interpreted 


‘0 Note that since all the students (including GSS students and potential comparisons) with complete matching 
data had credit and suspension outcome data, the final analytic samples for these two outcomes were exactly 
the same (with 348 matched pairs each). 


1! According to the WWC evidence standards, when a study uses a quasi-experimental comparison group 
design, the baseline equivalence must be established based on the final analytic samples without imputing 
missing data for outcomes or covariates (WWC, 2011). 
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cautiously, given the concern that hidden selection bias (e.g., resources, support) may influence 
outcomes. 


Methods Used for the Implementation Evaluation 


Information about implementation at the GSS schools was obtained from interviews, focus 
groups, surveys, school walk-throughs, and from documentation and data provided by GSS and 
the NYC DOE. 

e Site visits were conducted in the fall of each school year for the purpose of conducting 
individual interviews with the GSS and DOE administrators, focus groups with students 
and staff, and school walk-throughs to get a feel of the school climate. 

e All students were surveyed in December 2011, spring 2012, and spring 2013; the survey 
was developed by Metis to assess: the extent to which students feel supported by staff, 
relationships with peers, student leadership experiences, college and career preparation, 
educational aspirations, and satisfaction with their school. 

e Staff were surveyed in spring 2012 and spring 2013; the survey was developed by Metis 
to obtain staffs perceptions about implementation of the program model with regard to 
mission, the GSS and DOE partnership, support for students, student leadership 
opportunities, and college and career preparation. 

e Data on the characteristics of the students enrolled in the GSS schools were obtained 
from the NYC DOE. 

e Data on the GSS students’ participation in internships were provided by GSS. 


Information on practices in the comparison group transfer schools was obtained from interviews 
with some of the representatives of the community-based organizations partnering with or 
providing services to students in these schools as well as from information available to the public 
in the NYC DOE directory of high schools and on the department’s website. 


Strength and Weakness of the Implementation Study 


The implementation study used multiple sources and methods over a two-year period, allowing 
for a range of perspectives over an extended time. 


Response bias could have been present in the focus groups. Although student leaders as well as 
representatives from the different Advocate Counselor groups were involved in the focus 
groups, it is possible that students in the focus groups were more engaged in school, which may 
have shaped their responses. Staff focus groups included a mix of GSS and DOE staff. In 
consultation with GSS staff, each staff focus group was specifically designed to include this staff 
mix. It is possible that this combination created an atmosphere where staff were reluctant to talk 
about the shortcomings of the school in front of their colleagues from the other side of the 
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partnership. Only a small number of parents participated in focus groups; thus these data were 
rarely cited. 


For both student and staff respondents, the surveys presented no such bias. Students placed 
their completed surveys in sealed envelopes. Staff surveys were completed anonymously online. 
The staff survey response rate was 95% for each administration. 


Because of the logistics of starting any study at the beginning of the school year, the fall 2011 
student survey was not conducted until December and the response rate for one of the schools 
was low. The timing of the survey also meant that responses from this administration could not 
be considered baseline for newly-enrolled students; therefore the results of this survey are not 
presented. However, the response rates for the two spring survey administrations were 79.6% 
(spring 2012) and 73.1% (spring 2013), rates considered acceptable for this type of survey. 


Although Metis wanted to interview principals of the comparison transfer schools about their 
schools’ practices, this group was not responsive to requests. Instead, interviews were conducted 
with representatives of 5 community organizations in 5 of the 11 schools. The NYC DOE’s 
descriptions of the schools were also used to describe the schools; however the study would 
have benefitted from the availability of the same interview data for the comparison schools as 
was collected from the GSS schools. 


Methods Used for the Study of Youth Development 


An assessment of the developmental outcomes of GSS students was obtained through 
administrations of the Developmental Assets Profile (DAP), a tool developed by the Search Institute 
to assess the types of experiences and qualities that are viewed as essential to healthy 
psychological and social development of adolescents. These include external qualities such as 
support from and relationships with others, feelings of safety, and use of time as well as internal 
qualities such as motivation, responsibility and decision making, and locus of control (see 
Appendix E for details). School staff administered the DAP (along with the locally-developed 
student survey) in December 2011, spring 2012, and spring 2013, providing formative data to the 
schools. To assess differences over the course of a school year, the DAP was administered only 
to incoming students before the start of the 2012-13 school year (fall 2012) and again as part of 
the spring 2013 survey administration when all students were surveyed. A matched, longitudinal 
analysis was conducted on the fall 2012 and spring 2013 DAP responses from the group of 
students who were new to the school in 2012-13. 


Strength and Weakness of the Study of Youth Development 


The timing of the DAP administration has the same weakness as described above in relation to 
the timing of the student survey. However, the administration of the DAP to newly-enrolled 
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students was a true baseline, which allowed for a longitudinal analysis to assess any changes over 
the course of the year. 


A drawback of this analysis is the absence of a New York City high school student comparison 
group, an option that was explored but not feasible to implement. Instead, the Search Institute 
provided data for a sample of Dallas, Texas high school students that was administered the DAP 
in the spring of 2011. The sample consisted of more than 20,000 students whose demographic 
characteristics were similar to those of the GSS students, thus allowing for a comparison 
population (see Appendix H for details). 
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What is the Good Shepherd Services Transfer 


School Model? 


There is substantial support in the literature for the 
GSS transfer school model’s holistic approach to youth 
development and dropout prevention. In an extensive 
synthesis of the research and a review of various 
models that have been used to explain why students 
drop out, Rumberger (2011) proposes a framework that 
includes individual factors associated with students 
(background, attitudes, behaviors, and performance) 
and institutional factors that are associated with the 
contexts that influence students (families, schools, and 
communities). The author demonstrates the 
interrelatedness of these factors and the complexity of 
the reasons that students drop out of school. 

In adopting a model that approaches students as 
individuals, reaches out to family members, and seeks 
to engage students academically as well social- 
emotionally in a community-based environment, the 
GSS transfer school model recognizes the multiple 
influences on students. 


Furthermore, the GSS model includes programmatic 
approaches that have a moderate level of evidence for 
reducing dropout rates, including: 
e assigning adult advocates to students at risk of 
dropping out; 
e providing academic support and enrichment to 
improve academic performance; 
e personalizing the learning environment and 
instructional process; and 
e providing rigorous and relevant instruction to 
better engage students in learning and provide 


GSS transfer schools create a safe, 
caring, learning community for students 
that recognizes the different strengths of 
each student and family we work with. 
In partnership with the DOE, we 
work, with students to overcome 
obstacles to their success, mastering 
skills and Rnowledge necessary to earn 
a high school diploma and be college 
and career ready. Our work, both 


challenges and helps students and 


families in their journey to self- 


sufficiency setting them up for success in 
life after school. We define self- 
sufficiency as the ability to build 
healthy relationship, problem solve, set 
goals, navigate conflict and challenge, 
as well as motivation to change, 
effective communication skills, and 
taking responsibility for learning 
(Good Shepherd Services, 2007). 


the skills needed to graduate and serve them after they leave school (Dynarksi et al., 


2008). 


S) metis associates making a meaningful difference 1 


RE-ENGAGING YOUTH FOR HIGH SCHOOL SUCCESS 
WHAT IS THE GOOD SHEPHERD SERVICES TRANSFER SCHOOL MODEL? 


The daily practice of the GSS approach includes the following aspects: 
e Strength-based and transparent communication between students and staff 
e The value of a primary adult in each students’ life 


e Structured routines that provide clear, consistent expectations and ongoing feedback that 
promotes accountability both in and outside the classroom 


e Respect for student voice and a focus on youth participation and leadership development 


e A personalized learning environment where each student is known. 


Implementation of the Model 


The core components of the GSS model include: a defined target population and admissions 
process, partnership and shared leadership, integration of the advocate counselor within the 
school setting, personalized small school environment, and youth development approach to 
instruction. Although not listed as a core component in the description of the model, 
postsecondary preparation is another critical component. Each of these features is described 
below, along with evaluation findings related to their implementation. 


Information collected through interviews and available documentation indicate that, on the 
whole, the GSS transfer schools have many aspects in common with the schools in the 
comparison group, although there was a range of practices at these other schools. However, the 
GSS model offers a substantially different experience for students than traditional high schools. 
Highlights of the similarities and differences between the GSS transfer school model and schools 
from which comparison group students were drawn are presented in Appendix C. 


Admissions Criteria and Practices 


The GSS transfer schools are neighborhood-based; each has a defined geographic catchment 
area. The schools enroll students who are between the ages of 16 and 21, and who have dropped 
out of school or who have a history of truancy. The average student brings less than 15 credits 
upon arrival at the GSS schools and is two or more years over-age. 


The GSS schools conduct active outreach to recruit students who meet their admissions criteria 
through contacts with neighborhood high schools and NYC DOE guidance counselors. 
Following an admissions process that is designed to engage students, eligible students are 
interviewed by one of the schools’ advocate counselors (described below). An educational 
biography prepared by the advocate counselor serves as the basis for discussing the student’s 
educational future. Identification of a supportive adult—a parent, another family member, or 
another adult outside of the school—who can take an interest in supporting the student 
academically, is another part of the admissions process. The students’ commitment to their 
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education is affirmed in a letter signed by them, their parents, and the advocate counselors. New 


students attend a group orientation where the school’s core values and expectations are 
discussed. 


Partnership and Shared Leadership 


The schools are led by a NYC DOE principal and a GSS director in an equal partnership, with 
each responsible for different aspects of the school. Weekly full staff meetings and joint 
professional development support this shared approach. Furthermore, to reinforce the notion of 
a partnership, there should be no distinction by students and parents as to which staff are 
employed by GSS and which are employed by NYC DOE. 


Survey responses provide an indication of the staff's perceptions of the partnership as 
incorporating an active commitment of the DOE and GSS staff to work together (91% of staff) 
and in which these staff convey the same expectations to students and parents (75%). More than 
two-thirds of the staff believe their school is led equally by the DOE principal and GSS program 
director (67%) and that the leaders bring complementary skills to their roles (71%). 


Figure |. Staff perceptions of partnership and shared leadership, 2013 


DOE and GSS staff makes an active 
commitment to work together. 


DOE and GSS staff convey the same 
expectations to students and parents. 


The DOE principal and GSS program 
director bring complementary skills to 
their roles as school leaders. 


The DOE principal and the GSS program 
director lead the school equally. 


0% 20% 40% 60% 80% 100% 


Percentage of staff responding'agree' or 'strongly agree’ 
(N = 53) 


An analysis of the responses of teachers and advocate counselors shows that teachers are more 
likely than advocate counselors to agree or strongly agree that the school is led equally by the 
two administrators and that they bring complementary skills to their leadership roles, and that all 
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staff convey the same expectations to students and parents. However, the two groups of staff 
equally agreed that they make an active commitment to work together. 


Integration of Advocate Counseling 


Advocate counselors, as well as other adults in the schools, serve as a support system for 
students. Following a primary person model, each student is assigned an advocate counselor who 
is expected to continue with them until graduation.'? The role of the advocate counselor includes 
engagement of students and families, attendance outreach (including greeting students at the 
beginning of each day), social and emotional counseling, academic advisement and 
programming, and activities focused on the students’ postsecondary future and community 
building. In addition to meeting individually with students, the advocate counselors provide 
these supports through a required credit-bearing elective course. The class also serves as a group 
advisory and peer support group. Self-advocacy, responsibility, and communication, are major 
topics of discussion. 


The schools’ data reflect a high degree of success in achieving stability of this relationship. More 
than half of the students (61%) have had the same advocate counselor since they enrolled and 
most of the others have had just two advocate counselors (35%). On average, students stayed in 
the GSS schools for almost 22 months (including summer). 


In focus groups, GSS students contrasted the supports they received from the advocate 
counselors with the absence of such support at their previous high schools. At the GSS schools, 
they said, counselors are involved in every step of the process and available at all times to discuss 
academic and personal issues. Student survey responses also indicate that the school has helped 
them figure out the courses they need to graduate (88%) and develop career goals (78%), as 
shown in Figure 2. 


12 Assignment is based on individual needs as well as student characteristics. 
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Figure 2. Students’ report of academic advisement, 2013 


This school has helped me figure out 
what courses | need to graduate. (N 
= 229) 


This school has helped me develop 
career goals. (N = 226) 


0% 20% 40% 60% 80% 100% 


Percentage of students responding 'somewhat true’ 
or ‘completely true’ 


Personalized, Small School Environment 


As a small school with small class sizes, the GSS transfer schools are designed to offer a high 
level of individual attention. Enrollment at each school ranges from 150 to 200 students. Classes 
have approximately 18 students, with a maximum of 25 students per class. 


The size of the school and its classes were among the reasons GSS students offered for wanting 
to enroll in the schools. While in large schools, a student commented, it was easy to “get lost in the 
pack,” but at the GSS schools, “people care about what happens to [students].” The schools break down 
barriers between staff and students by using the staffs first names while modeling mature 
relationships. One staff member commented, “We're not just teachers, we're educators; we educate them 


about life.” 


In focus groups, students described the relationship between staff and students as “familial” and 
commented that they “fee/ at home” in the school. They added that they were comfortable going 
to staff members and were close to staff because “they know they will be heard and not rejected.” In the 
survey, 83% of students agreed that there are adults they can turn to if they have a problem or 
want to talk. As shown in Figure 3, 98% of staff also agreed. 
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Figure 3. Perceptions of staff support of students, 2013 


Staff: There are staff at this school that 
students can go to if they have a problem or 
want to talk. (N = 53) 


Students: There are adults at this school 
that | can go to if | have a problem or want 
to talk. (N = 233) 


0% 20% 40% 60% 80% 100% 


Percentage responding ‘agree’ 
or ‘strongly agree’ 


Youth Development Approach to Instruction 


Teachers use strategies that promote a high level of active participation, including the workshop 
model, differentiated instruction, and hands-on learning. The schools also work to engage 
students in learning by offering opportunities for them to voice their opinions and through 
teamwork. Students have a personalized course 


schedule based on the courses and Regents exams You can be a helper just as much as the 
they need to graduate. Because they are each on their person who needs help. [Students] learn a 
own schedule, students are not organized according sense of ‘belongingness,’ agency, developing a 
to a grade level, but are in ungraded cohorts. voice. It’s empowering. 


Academic expectations are clear and every teacher has 
the same standards and grading policy. —GSS Transfer School Leader 


Students get regular feedback about their academic progress through biweekly meetings and 
benchmark reports—a two-page document that gives an overview of a student’s progress in each 
course. According to one staff member, “The frequent assessment of academics allows us to provide early 
intervention to students. Benchmark conversations help zero in on the issues and promote goal-setting.” In focus 
groups, students reported that the benchmarks allow them to see their progress or shortfalls 
every two weeks and the connection between grades and attendance. A student commented, 
“Benchmarks make the missed work real.” 


Student feedback is sought through formal and informal means, including in student council 
meetings, group meetings with advocate counselors, surveys, and end-of-course evaluations. As 
an example, student input about the blended learning model that had been adopted was obtained 
through focus group meetings, and changes were made for the subsequent cycle. 
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Levels of student engagement are evident in student survey responses, in 75% of students 
reported that they have been an active participant in discussions, 57% have led an activity, and 
50% have contributed solutions for a community problem. Additional student responses are 
presented in Figure 4. The importance the schools place on these types of student involvement 
is reflected in staff survey responses, shown in Figure 5. 


Figure 4. Student engagement in a leadership role, 2013 


Worked as part of a team (N = 231) 73% 


Been an active participant in discussions (N = 231) 


Been asked by staff or other participants for my ideas 
about the program or an activity (N = 231) 


Led an activity (N = 230) 57% 


(N = 230) 


Contributed solutions for a community problem [awe S0N | 
2 - «fF — . oe = uw oc; of 


Percentage of students responding'yes' 


Figure 5. Staff perceptions of importance of student leadership, 2013 


Work as part of a team ee 
Be an active participant in discussions | pu 
Contribute solutions fora community problem 87s] 
Lead an accivicy 578 
Be asked by staff or other participants for ideas. a 
Help establish rules for students ee 
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Percentage of staff responding 'very important’ (N = 53) 


& metis associates making a meaningful difference 17 


RE-ENGAGING YOUTH FOR HIGH SCHOOL SUCCESS 
WHAT IS THE GOOD SHEPHERD SERVICES TRANSFER SCHOOL MODEL? 


In focus groups, staff noted that some opportunities for students to exhibit leadership were 
postponed or curtailed, in part because of a major storm (Superstorm Sandy) that occurred in 
the fall of 2012. Although both schools were affected, the storm severely affected one school 
and the lives of students living in the surrounding community. In addition to being closed 
entirely for several days, the school had to consolidate the use of its space during a three-month 
renovation and its cafeteria served as a donation distribution and food refrigeration site for the 
neighborhood. Indeed, a comparison of student survey responses from spring 2012 to spring 
2013 shows small declines in the percentages of students who have been engaged in the types of 
activities described above. 


Group staff activities, including full staff meetings, inquiry teams, case conferences, and retreats, 
were among structures in place at the school for facilitating the partnership and developing the 
staffs capacity to implement a youth development model. Ratings of how well the schools have 
oriented staff to the model and provided professional development or training to staff are 
presented in Figure 6. More than half of the staff rated these aspects ‘good’ or ‘excellent.’ 
However, more than a third of the staff gave ratings of ‘adequate,’ ‘fair,’ or ‘poor,’ suggesting a 
need to further develop staffs’ capacity to implement the model. Advocate counselors were more 
likely than teachers to rate these aspects as ‘poor’ or ‘fait.’ 
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Figure 6. Staff ratings of orientation to the model and professional development, 
2013 


Providing professional development/training 


for school staff 8% 8% 19% 43% 23% 


Orienting school staff to the youth 


development model 9% 15% | 17% 38% 21% 
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Percentage of staff (N = 53) 


™Poor MFair ®Adequate ®Good m® Excellent 


Furthermore, the ratings were lower in the 2013 survey compared to the 2012 survey. Over this 
period, the percentage of staff that rated the professional development as ‘good’ or ‘excellent’ 
declined from 70% to 66%. Similarly, 59% of staff rated the schools’ orientation of staff to the 
youth development model as ‘good’ or ‘excellent’ in 2013, down from 67% in 2012. 


Postsecondary Preparation 

The GSS transfer school model is designed to help students master the skills and knowledge 
needed to earn a high school diploma and—focusing on the future—be college and career ready, 
and promote self-sufficiency. The importance of college and preparation for the future is 
communicated at intake and throughout a student’s life at the school, with an emphasis on 
students’ learning to take responsibility for their actions and make their own decisions. 


Teachers and advocate counselors team up for postsecondary planning which is part of the 
required elective class, described above, and each school has a full-time college 
advocate/advisor. Most students take an introduction to job readiness or internship readiness 
class. The schools also offer the Learning to Work program, sponsored by the NYC DOE’s 
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Office of Postsecondary Readiness.!3 The program supports paid internships as well as pre- 
internship workshops and weekly seminars during the internship. Advocate counselors and 
teachers work with students to help them develop a postsecondary plan. As students get closer 
to graduation, they take a class on college and careers. 


Survey data indicate that the GSS transfer school experience includes learning about 
postsecondary options and what is needed to be successful after high school. As shown in Figure 
7, more than three-quarters of the students have learned about colleges and job skills. 
Furthermore, data provided by GSS indicate that 64% of the students enrolled during the study 
period had participated in an internship. 


Figure 7. Postsecondary preparation, 2013 


This school has taught me about colleges. 
(N = 228) 


This school has taught me about job skills. 
(N = 230) 


This school has taught me how to find a job. 
(N = 228) 


This school has taught me how to prepare for 
college or a vocational program. (N = 228) 


This school has taught me how to interview 
for a job. (N = 227) 


This school has taught me how to complete 
college applications. (N = 227) 
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Percentage of students responding 'somewhat true’ or ‘completely true’ 


13 Learning to Work is also the NYC DOE funding stream for services provided by community-based 
organizations to transfer schools. The funding supports some of the GSS staff members (e.g., advocate 
counselors, internship coordinator, a portion of the director) and the internship stipends. 
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Summary of Implementation 


The students enrolled in the GSS transfer schools fit the population the model is intended to 
serve. Qualitative data obtained through surveys, interviews, and focus groups indicate that the 


principles and components of the model are being implemented as designed. Students reported a 


high level of satisfaction and staff reported a high level of adherence to the schools’ mission. 


The GSS transfer school model expects to provide students with an experience that will help 


them to develop academically so that they graduate 
from high school, but also support other aspects of 


their development—socio-emotional, life skills, college, 


and careers. Asked to rate these aspects of their school, 
a majority of staff gave high ratings (good or excellent) 
in every area, although there is room for improvement 
in encouraging student leadership and postsecondary 
goals (preparing students for college and for 
adulthood). 


Figure 8 presents the staff ratings of their school, 
including school climate, supporting students 
academically and social-emotionally, and preparing 
students for adulthood. 


© 86% of students were ‘satisfied’ or 
very satisfied’ and 88% indicated 
that they would recommend their 
school to a friend in similar 
circumstances. 


77% of staff ‘agree’ or ‘strongly 
agree’ that the schools’ mission is 
reflected in the everyday experiences 
at their school. 
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Figure 8. Staff ratings of their school, 2013 


School culture and climate ] 21% 36% 38% 
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Monitoring students' academic progress [/:)7 55% 38% 
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The outcome study examined the impacts of the GSS model on students’ academic and 
behavioral outcomes. After matching one-to-one the target GSS students to students in the 
comparison transfer schools, analyses'* were conducted for the following outcomes: 

1) graduation; 

persistence in school; 

credits earned; 

core Regents exams passed; 

school attendance; and 


DIB WN 


school suspensions. 


Anticipating that graduation outcomes for the transfer school population are typically longer 
term, Metis analyzed GSS and comparison student data on graduation and enrollment 
(persistence) outcomes after an additional school year. Therefore additional analyses to examine 
graduation, and persistence, were undertaken once data for the 2013-14 school year were 
available. All of the other outcomes listed above were analyzed through 2012-13. 


When the number of students was sufficient, additional analyses were conducted for 
demographic subgroups of students: 

(1) females; 

(2) males; 

(3) Black males; 

(4) Hispanic males; and 

(5) students with less than 11 credits at admission.!5 


4 Analyses included hierarchical linear models, hierarchical generalized linear models (a parallel form of logistic 
regressions for dichotomous outcomes using multi-level modeling), multiple linear regressions and binary 
logistic regressions. 


15 Note that all findings for subgroup analyses were exploratory in nature since baseline equivalence might not 
be established for pertinent analytic samples that contained only subgroups of original matched youth. 
Therefore, no multiple comparisons adjustment (e.g., Bonferroni, Benjamini-Hochberg) was applied for any 
outcomes in this study—the overall analysis results were considered confirmatory. 
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Findings for both the overall and subgroup analyses are presented for each of the intended 
outcomes. For all analyses, the effect size and statistical significance are reported. Specifications 
of full models and detailed regression analysis results for the 2011-13 analysis can be found in 
Appendix D. Similar information for the 2011-14 analysis can be found in Appendix E.!6 Note 
that unless otherwise stated, the outcome findings are regression adjusted. That is, the presented 
rates, means, etc., ate not observed, but rather estimates from which the effects of other 
covariates have been factored out through statistical modeling. 


Graduation 


Students’ graduation vs. discharge status by the end of the 2012-13 and 2013-14 school years 
was examined.!7 Although not statistically significant at the end of the 2012-13 school year, by 
2014 the results showed that the GSS students were more likely to graduate from high 
school than comparison students. 


As displayed in Figure 9, on average, at the end of 2013-14, 63.0% of the GSS students were 
expected to graduate, while it is projected that only 51.4% of the matched comparisons would 
have this outcome. According to the WWC standards, the effect size for the finding at the end 
of 2013-14 (0.287) is substantively important, whereas that measure at the end of 2012-13 is not 
(0.234).18 


'6 Following the WWC requirements for quasi-experimental studies, Metis did not impute missing data for 
outcomes or predictors in these analyses. 


'7 Because the outcome measute is (yes/no), Metis used logistic and multi-level logistic regressions for the 
analyses. See Appendix D and Appendix E for full model specifications. 


18 For logistic or multi-level logistic regressions of dichotomous outcomes (ie., graduation vs. discharge status, 
enrollment vs. discharge status), the WWC (v 2.1, 2011) adopts the Cox index as the default effect size 
measure. According to the WWC (vy 2.1, 2011), a Cox index of 0.25 or larger is considered to be substantively 
important, regardless of whether it reaches statistical significance. 
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Figure 9. Overall impact of GSS model on graduation 


Regression-Adusted Probability 


GSS students Matched comparison GSS students Matched comparison 
students students 
2012-13 school year (N = 225 x 2) 2013-14 school year (N = 291 x 2) 


*Statistically significant, p = <.05. 


Of further interest, analyses from both years showed that female GSS students had a 
significantly higher probability of graduating than the female comparison students.!° 


As seen in Figure 10, on average, it is anticipated that 73.7% of the female GSS students would 
graduate at the end of 2012-13, while this is projected for only 45% of the female comparisons. 
By the end of 2013-14, the difference in the proportion of female graduates between the two 
groups was closer, but nonetheless still statistically significant; 67.8% of the female GSS students 
graduated compared to 53.2% of the comparisons. According to the WWC, this effect is 
substantively important for both years (0.742 and 0.374, respectively). 


1° Note that all findings based on subgroup analyses should be interpreted with caution, since the baseline 
equivalence might not be established for pertinent analytic samples that contained only subgroups of original 
matched students (i.e., students in a matched pair may not both be included in a given subgroup analysis). 
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Figure 10. Impact of GSS model on graduation of female students 
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GSS students Comparison students GSS students Comparison students 


2012-13 school year (N = 230) 2013-14 school year (N = 298) 


*Statistically significant, p = <.05. 


Mirroring the results obtained for overall graduation rates, no statistically significant difference is 
observed in expected graduation rates between the GSS and comparison student groups 
admitted with less than 11 credits by the end of the 2012-13 school year. However, by the end 
of the 2013-14 school year, statistically significant higher percentages of GSS students 
were expected to graduate than the comparisons. As shown in Figure 11, on average, 52.6% 
of the students at the GSS schools admitted with less than 11 credits were likely to graduate 
compared to 35.6% of comparison students belonging to the same subgroup. Note that the 
effect size associated with this finding (0.423) is considered substantively important by the 
WWC. 
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Figure | 1. Impact of GSS model on graduation of students with less than || credits 
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*Statistically significant, p = <.05. 


Finally, higher percentages of GSS Hispanic males were expected to graduate than their 
comparisons in both years, although neither finding is statistically significant (see Appendix 
Tables D.7 and E.3). 


The results for the analysis of male students were mixed, although not statistically significant in 
either year. Analyses were not conducted for the Black male subgroup due to insufficient sample 
size. 


Persistence in School 


Students’ status in school, whether they remained enrolled vs. whether they had been discharged, 
served as a measure of persistence in school, and was examined for the two time periods.”° In 
each year, although the GSS students overall were more likely to remain in school than 
the comparison group, the differences are not statistically significant. 


As shown in Figure 12, on average, 56.4% of the GSS students were expected to stay enrolled in 
school at the end of 2012-13, and 30.9% of them were expected to remain at the end of 2013— 


20 Note that this outcome measure was dichotomous (yes/no), so logistic or multi-level logistic regressions 
were used for the analyses. See Appendix D and Appendix E for full model specifications. 
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14, whereas 50.8% of the matched comparisons were expected to remain in school the first year, 
and 26.3% in the second year. The GSS schools’ effect on persistence in school is considered 
small (0.136 and 0.135, respectively) and not substantively important based on the pertinent 
effect size measure. It should be noted that, over time more individuals are moving toward a 
terminal outcome thus reducing the size of the group for which persistence data?! are available. 


Figure 12. Overall impact of GSS model on persistence in school 


Regression-Adjusted Probability 


Oo as 
GSS students Matched comparison GSS students Matched comparison 
students students 


2012—13 school year (N = 200 x 2) 2013-14 school year (N = 155 x 2) 


All of the subgroup analyses for persistence also did not achieve statistical significance 
(See Appendix Tables D.6 and E.2). However, it should be noted that for both 2012-13 and 
2013-14, a higher percentage of female GSS students were expected to persist in school 
than the comparison students. While not statistically significant, the effect for this group is 
substantively important (0.280 in 2012-13 and 0.339 in 2013-14). Conversely, for males, the 
results across both years anticipated a lower percentage of GSS persisters than the 
comparisons—although the effect is small at best. 


Finally, the directions of observed differences in persistence between GSS and comparison 
students for the Hispanic males and students with less than 11 credits changed from year to year, 
(and the magnitudes were very small), suggesting little to no effect. The analysis for the Black 
male subgroup was not conducted due to insufficient numbers. 


71 Note that persistence is being measured by current enrollment in school. Those individuals who do not have 
a discharge code are the members of the persistence analytic sample. As more individuals receive terminal 
discharge codes, the persistence group shrinks. 
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Credit Attainment 


Credit attainment was measured by the number of credits students earned after they were 
admitted to the transfer schools through the end of the 2012—13 school year.” The overall 
analysis result shows that the GSS model had significant positive impact on credit 
attainment. After enrolling in the transfer schools, the GSS students on average earned 4.3 
more credits than their matched comparisons (20.6 vs. 16.3, see Figure 13). The corresponding 
effect is of medium size (0.37, see Appendix Table D.1) and substantively important according 
to the WWC.%3 


Figure 13. Overall impact of GSS model on credit attainment 
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*Statistically significant, p = <.05. 


2 Note that all credit attainment analyses controlled for the number of credits earned before enrolling in the 
transfer schools, in addition to the predictors included in the analyses of other outcomes. 


3 Effect sizes for continuous outcomes included in this report (i.e., number of credits earned and core Regents 
exams passed, attendance and suspensions) were derived based on Glass’s Delta. A meta-analysis of 186 
education intervention studies indicated that the bottom third of studies ranged from 0.00 to 0.32, the middle 
third from 0.33 to 0.55, and the top third from 0.56 to 1.20 (Lipsey, 1990). These ranges could help loosely 
define small, medium, and large effects. According to the WWC standards (v 2.1, 2011), for continuous 
outcomes, effect sizes of 0.25 standard deviations or larger are considered to be substantively important, 
regardless of whether they reach statistical significance. 
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The majority of the subgroup analysis results of credits earned after admission was 
consistent with the conclusion drawn from the overall impact analysis (see Appendix 


Table D.4). 


e The female GSS students on average accumulated 5.5 more credits than the female 
comparison students (21.7 vs. 16.2), with a medium to large effect size (0.48). This finding 
was statistically significant. 


e The GSS male students on average accumulated 2.7 more credits than the comparison group 
(19.1 vs. 16.4); although the effect size is relatively small (0.23). This finding was statistically 
significant. 


e The GSS Hispanic male students on average accumulated 3.0 more credits than the 
comparison group (18.8 for GSS and 15.8 for comparison), with a substantively important 
effect size (0.26). This finding was statistically significant. 


e Among students with less than 11 credits at admission, the GSS group on average 
accumulated 5.3 more credits than the comparison group (21.5 for GSS and 16.2 for 
comparison), with a medium effect size (0.41). This finding was statistically significant. 


GSS Black male students, however, showed a very small difference, on average, in accumulated 
credits (17.4 vs. 17.0), and the corresponding effect is very small (0.03). This finding was not 
statistically significant, in part because it was based on a small number of observations. 


Core Regents Exams 


In addition to earning 44 credits, students must pass five core New York State Regents exams?* 
to graduate. Therefore, the number of core Regents exams passed after admission serves as an 
important academic outcome measure. Students may take these exams at any point during high 
school (and can repeat the exams if they do not score a passing grade). The overall analysis 
result indicates that the GSS students did not show any significant difference in the 
average number of core Regents exams passed after admission, when compated to 
students in the comparison group. 


As seen in Figure 14, by the end of the 2012-13 school year, both the GSS students and their 
matched comparisons passed approximately one and a half core Regents exams after being 
admitted to transfer schools.?5 


4 The five required NYS Regents exams are Comprehensive English, Global History, U.S. History, a math 
exam (usually Algebra), and a science exam (usually Living Environment/Biology or Earth Science). 


°5 Note that the number of students with available Regents outcome data was substantially smaller than earlier 
analyses (N = 99 pairs). The core Regents exams analyses were restricted to those who were still enrolled in 
school (i.e., those who had graduated or discharged were excluded from this analysis). In addition to the 
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Figure 14. Overall impact of GSS model on number of core Regents exams passed 
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The subgroup analyses for number of Regents passed are mixed and none of the 
findings achieved statistical significance (see Appendix Table D.5). 


e For females and students who had less than 11 credits at admission, the average number of 
core Regents exams passed after admission for the GSS students was a little smaller than that 
for the comparison group and both impacts were estimated to be of small size (0.15 and 0.23 
respectively). 


e Among males and among Hispanic males, however, the GSS group passed more Regents 
exams than the non-GSS group. Note that for the subgroup of Hispanic males, the effect is 
substantively important (0.29), while that for males is a little smaller (0.20). 


The analysis for the Black males was not carried out due to an insufficient number of cases. 


predictors included in the analyses of other outcomes, these analyses also controlled for the number of core 
Regents exams passed prior to admission. 
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School Attendance 


The school attendance outcome was measured by students’ average daily attendance (ADA) over 
two school years, 2011-12 and 2012-13. The result of this analysis indicates that the GSS 
students had significantly better school attendance than their matched comparisons at 
the end of 2012-13. 


As shown in Figure 15, the GSS students had a 65.4% attendance rate, outperforming 
comparison students by 5.4 percentage points, after controlling for a number of student and 
school characteristics.7° The effect size calculated for this finding (0.25, see Appendix Table D.1) 
is considered substantively important by the WWC. 


Figure 15. Overall impact of GSS model on school attendance 
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*Statistically significant, p = <.05. 


Most of the results of the subgroup analysis for school attendance during the two school years 
were consistent with those of the overall impact analysis (see Appendix Table D.2). Thus, 
female as well as male GSS students, Hispanic males, and students who had less than 11 
credits at admission, each outperformed their comparison group in terms of attendance. 


e The female GSS students had a higher average daily attendance rate than the comparison 
students (64.3% vs. 59.5%). This finding was statistically significant. 


6 Linear regressions were used for all ADA analyses since the between-school variance did not converge when 
fitting HLM models for overall or subgroup analyses (see Appendix D for details). 
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e The male GSS students had a higher average daily attendance rate than the comparison 
students (66.5% vs. 60.6%). This finding was statistically significant. 


e The Hispanic male GSS students had a higher average daily attendance rate than the 
comparison students (66.4% vs. 59.5%). This finding was statistically significant. 


e Among students with less than 11 credits at admission, GSS students had a higher average 
daily attendance rate than the comparison students (64.1% vs. 55.5%). This finding was 
statistically significant. 


Note that the effect sizes calculated for the analyses of males, Hispanic males and those with less 
than 11 credits at admission all reach substantive importance (see details in Appendix Table 
D.2), while that for females falls just short of that level (0.21). 


For Black males, the difference between GSS and comparison students (63.9% vs. 61.4%) did 
not reach statistical significance and the corresponding effect size is small (0.12), likely due to 
insufficient statistical power as there were only a very small number of Black males included in 
the analysis. 


School Suspensions 


With limited data available on student behaviors, the number of school suspensions over two 
yeats was used as a measure of behavioral outcomes. Unlike the finding for school attendance, 
the overall analysis result of suspensions showed that the GSS model did not have an 
impact on this outcome. 


As displayed in Figure 16, the average number of suspensions per student for the GSS and the 
matched comparison group for the combined 2011-12 and 2012-13 school years were almost 
identical after controlling for various student and school characteristics (0.128 for GSS and 0.129 
for comparison). However, only 24 of 348 GSS (3 more than once) and 40 of 348 comparison (9 
more than once) students were suspended during the two-year period. Thus, the vast majority of 
each group never experienced a suspension. 
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Figure 16. Overall impact of GSS model on school suspensions 


Regression-Adjusted Means 


GSS students (N = 348) Matched comparison students 
(N = 348) 


The subgroup analyses of suspensions during the two school years showed more 
vatiations in the estimated GSS-comparison differences, but the conclusions were the 
same as the overall analysis: the GSS students did not show any significant differences in their 
average number of suspensions from the comparison group (see Appendix Table D.3). 


While the result for Hispanic males indicates that the GSS students had more suspensions than 
the comparison students, the analyses of the other subgroups (including females, males, Black 
males, and students with less than 11 credits at admission) showed the opposite—the average 
number of suspensions was smaller for the GSS students than for the comparisons. Based on 
these analyses, only the female subgroup show a substantial effect size (0.20), whereas the effects 
for the other subgroups are very small (see Appendix Table D.3). 


Again, similar to the overall analysis, average numbers of suspensions for all the subgroups, 
either GSS or comparison students, were low (< 0.2), and the estimated program impacts are 


negligible (all < 0.1). 


Summary of Impact 


The findings from these overall impact analyses indicate that when compared to the matched 
comparisons, the GSS students have a significantly better chance of graduating, a higher 
average rate of school attendance, and also earn significantly more credits after 
enrollment—all effects are substantively important according to the WWC. In terms of 
graduation, GSS students outperformed comparison students by 12 percentage points, 63% to 
51.4%. In comparison, the six-year graduation rate for over-age and under-credited students in 
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transfer schools citywide (Class of 2013) was 41%, while for OA-UC students in traditional high 
schools it was only 29% (NYC DOE, 2014). 


The observed difference between the GSS students and the comparison students in persistence, 
core Regents exams passed, and suspensions was negligible and not statistically significant. 


Regarding the exploratory subgroup analyses, the most notable findings were that female GSS 
students and GSS students who enter the transfer schools with less than 11 credits have a 
significantly higher probability of graduating than the comparison students. 


Other significantly positive findings for school attendance and credits earned after admission 
were found for the subgroups of females, males, Hispanic males, and those with less than 11 
credits at admission. All the other subgroup analyses, however, did not show statistically 
significant results. 
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To address the research question, “What are the youth development outcomes of GSS transfer 
school students and what is the relationship between these outcomes and academic performance 
or other characteristics?” the evaluation included an assessment of developmental assets as 


measured by the Developmental Assets Profile (DAP). Developed by the Search Institute, the DAP is 
designed to assess how youth are faring personally and socially. 


Developmental Assets are defined as positive experiences and qualities identified by the Search Institute 
as being essential to healthy psychological and social development in childhood and adolescence. These 
assets have the power to influence young people’s developmental trajectories, protect them from a range of 
negative outcomes, and help them become more productive, caring, and responsible adults. Developmental 
assets represent the positive relationships, opportunities, skills, and values that promote the positive 
development of all children and adolescents (Haggerty et al., 2011). 


The DAP asks young people about the frequency or intensity with which they have experienced, 
“now or within the past three months” a list of 58 “positive things that you might have in 
yourself, your family, friends, neighborhood, school, and community.”’?’ The items are grouped 
into 40 developmental assets, with half categorized as external assets or strengths and half 
categorized as internal. The external assets/strengths assess a youth’s experiences and 
relationships with peers, parents, teachers, and other adults in their community. Internal 
assets/strengths include those qualities that young people develop gradually as they become 


more self-regulating (Search Institute, 2005).28 Figure 17 provides a description of the subscales 
for both the External and Internal DAP categories.”? 


Figure 17. Description of DAP categories and subscales 


Parent-adolescent communication, family support, as well as caring, 
Support encouragement, and support outside the family from the 
neighborhood, school, and community 


External 


A general feeling of safety across many contexts; feeling valued, 
Empowerment 
useful, and respected by others 


27 The questions are asked in a series of “I” statements, with a four-point response scale: not at all or rarely, 
somewhat or sometimes, very or often, and extremely or almost always. 


8 See Appendix F for further discussion of the DAP and the developmental assets it measures. 


?° For additional information on interpretation, see Appendix F, Figure F.2. 
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Parental support; safety in a variety of settings; rules and 
Boundaries and Expectations | consequences in a variety of settings; and role models among 
friends, family, and outside the family 


Involvement in extracurricular activities in one of four areas: (1) 
Constructive Use of Time religious or spiritual; (2) sports, clubs, or other groups; (3) creative 
arts; and (4) family life 


Motivation and rewards related to learning and active engagement 
Commitment to Learning in learning, both tied directly to school and extending outside of 
school 


Personal virtues such as honesty, integrity, responsibility, and 
Positive Values restraint, as well as caring about others and working for equality 
and social justice 


Internal 


Planning and decision making, cultural competence, and social skills 
Social Competencies involving the ability to build friendships, resist negative peer 
pressure, and resolve conflicts peacefully 


Adolescent’s emerging identity, including self-esteem, internal locus 


Hoste [eeuaiy) of control, optimism, and sense of purpose in life 


Research suggests that youth who report relatively more assets are more likely to report engaging 
in positive, socially constructive behaviors and less likely to report engaging in risky behaviors 
(Benson et al., 2011). Analyses conducted by the Search Institute also have found that a higher 
level of developmental assets is associated with academic achievement as measured by grade 
point average (Scales et al., 2006) and with better college- and career-readiness outcomes. 3? 
Thus, it is worthwhile for a program to focus attention on increasing students’ assets, in order to 
have a greater impact on academic achievement and future college success. 


The DAP was administered to new students just prior to their enrollment in the GSS schools in 
the fall of 2012 and to all students in the spring of 2013. The analysis presented in this report 
was designed to assess whether there were any changes in assets among the group of new 
students for which both pre- and post- scores were available (N=75). The students’ scores in the 
fall represent a baseline level of assets that could, over time, be expected to be influenced by 
their experience at the GSS schools. The analysis also identified variables (demographic, 
academic, student engagement) that predicted these students’ end-of-year scores. 


5° College- and career-readiness outcomes are defined as key cognitive strategies (e.g. problem solving, 
reasoning, communication) in core subject areas; students’ time management and study habits; students’ 
perceptions of the academic rigor of their core classes; the degree to which students are engaged in 
researching colleges; and the level of support students report receiving from school and family in learning 
about colleges and how to apply (Search Institute, 2012). 
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The DAP survey items are asked within a three- 
month time frame, with respondents instructed 
to describe themselves “now or within the past 
three months,” thus the re-administration of the 
DAP nine months after the first administration 
falls within the Search Institute’s guidelines to 
detect meaningful changes. It should be 
recognized, however, that this is a short period 
to effect change and it is possible that the 
disruption caused by Superstorm Sandy early in 
the school year had a negative effect on 
students’ reports of the extent to which they felt 
supported and engaged during this time period. 


Figure 18 shows the level of developmental 
assets reported by the GSS students in the fall 
and spring. The results show a positive trend, 
with an increase in the percentage of students 
considered ‘thriving’ and a small decrease in the 


DAP scores are interpreted nithin four 
levels: low, fair, good, and excellent. Low 
scores indicate a depleted level of assets and 
tremendous opportunity for strengthening 
assets in most areas. Scores in the fair’ 
range indicate a borderline level, with many 
assets weak and/or infrequent. Within this 
level, there is considerable room for 
strengthening. Scores in the ‘good’ range 
suggest a moderate level of assets, still with 
room for improvement. An excellent level 
indicates that most assets are experienced 


strongly and/or frequently. 


percentage of students considered ‘at risk.’ However, results for the two middle groups were 


mixed, with an increase in the percentage of students who were ‘vulnerable’ and a decrease in the 


percentage of students with a ‘healthy’ level of assets. 


Figure 18. Level of GSS students’ Developmental Assets, fall 2012 and spring 2013 
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Several of the DAP subscales offer some insight into areas pertaining to the GSS model, 
although the DAP is not solely focused on the school environment and experiences. The 
external domains in which students improved included feelings of empowerment (which 
includes having useful roles and responsibilities and feelings of safety) and constructive use of 
time (involvement in sports and arts, as well as religious activities). 


e With regard to Empowerment, the percentage of GSS students whose responses were in 
the ‘excellent’ range increased from 20% to 33%, with the shift occurring from ‘good’ to 
‘excellent,’ while the ‘fair’ and ‘low’ ranges were stable. Note, however, that the 
Empowerment subscale includes items unrelated to school, such as feeling safe at home 
and in the neighborhood and inclusion in family decisions. 

e With regard to Constructive Use of Time—the percentage of students whose responses 
were in the ‘excellent’ range increased from 9% to 19%, while the percentage of students 
in the ‘low’ range decreased from 49% to 39%; 


Students’ feelings of support decreased, although it should be noted that several of the items that 
make up this subscale relate to support from parents, family members, and neighbors. However, 
this subscale also includes support from other adults and a school “that cares about kids and 
encourages them.” In this area, the percentage of students with assets in the ‘good’ range fell 
from 27% to 16%, while the percentage of students in the ‘low’ range rose from 9% to 15%. 


For the most part, students’ scores in the internal domains—comprised of items that assess 
students’ commitment to learning, positive values, personal responsibility and decision-making, 
and locus of control—decreased. For example, on the Commitment to Learning subscale, the 
percentage of students scoring at a ‘good’ level decreased from 25% to 15% and the percentage 
scoring at the low level increased from 13% to 25%. Commitment to Learning includes caring 
about school, enjoyment of learning and reading, and a desire to do well in school. 


Figure 19 presents a longitudinal analysis of the distribution of responses in fall 2012 and spring 


2013 for each subscale that comprise the external assets and Figure 20 presents the same 
information for each internal assets subscale. 
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Figure 19. GSS students’ external assets by subscale, fall 2012 and spring 2013 
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Figure 20. GSS students’ internal assets by subscale, fall 2012 and spring 2013 
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Drilling down further into these findings, one can find a number of specific DAP items where 
there was a substantial change (at least 10 percentage points) from fall to spring.*! Within the 
category of external assets, shown in Figure 21, three of the four items in which there was 
substantial change (all in a positive direction) are part of the Constructive Use of Time subscale. 
Two of the items (involvement in sports or clubs, involvement in the arts) may have been 
affected by the students’ participation in GSS school activities. It is also possible that the item 
from the Boundaries and Expectations scale, “Friends who set good examples for me,” could have also 
been influenced by their personalized learning experiences at the GSS schools, such as small 
school and class size, family-like atmosphere, and community building activities. 


Figure 21. External assets items with change from fall 2012 to spring 2013 


Items with a difference of at least 10 percentage points, fall 2012 to spring 2013 


Friends who set good Fall 2012, 60% Boundaries 
examples for me. Spring 2013, 71% and Expectations 


Involved ina religious Fall 2012, 28% 
group or activity. Spring 2013, 45% 


Involved ina sport, Fall 2012, 47% Constructive 
club, or other group. Spring 2013, 57% Use of Time 


Involved in creative Fall 2012, 49% 
things such as music, : = 
theatre, or art. ee eee 
T T T T 1 


0% 20% 40% 60% 80% 100% 


Percentage of students responding ‘very or often’ and ‘extremely or almost always’ 


Within the category of internal assets, shown in Figure 22, five items showed substantial change, 
although three were in a negative direction. Within Commitment to Learning, while there was a 
decrease in the percentage of students who said they care about school (possibly reflecting an 
end-of-year impatience or realization about the hard road ahead these students face), more 
students reported that they enjoyed reading, a finding that may be related to their classroom 


5! Complete responses are presented in Appendix F, Tables F.2 and F.3. 
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experiences at the GSS schools, where teachers work to provide authentic learning 
opportunities. The reason for these disparate results is not readily apparent, but may, as noted 


above, reflect discouragement about how far behind the students feel after their first year in the 
GSS schools. 


Serving others in the community also saw an increase, which could be reflective of the schools’ 
civic engagement activities or the personalized learning environment in which students feel a 
connection. The negative findings related to substance use and conflict resolution may reflect the 
larger community influences these students experience, and could point to a need to marshal 
additional resources to address these areas. 


Figure 22. Internal assets items with change from fall 2012 to spring 2013 


Items with a difference of at least 10 percentage points, fall 2012 to spring 2013 


Enjoy reading or Fall 2012, 45% 
being read to. Spring 2013, 55% 
4 Commitment 
to Learning 
Fall 2012, 81% 
Care about school. - 
Spring 2013, 71% 


Stay avay om Fall 2012, 65% 

tobacco, alcohol, See ET 
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4 Positive 
: Values 
Serving others in Fall 2012, 40% 

my community. Spring 2013, 51% 


Resolve conflicts 
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without anyone 2 = Social — 
getting hurt. Spring 2013, 73% Competencies 
1 


T T T T 
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Percentage of students responding 'very or often’ and ‘extremely or almost always’ 


The slightly smaller difference for the Social Competencies item represents a rounding error. 


Further analyses were conducted to identify what predicted the end-of-year DAP scale scores of 
these new students. The variables in the analyses included student scores on the fall 2012 DAP, 
demographic and educational characteristics, and school attendance and credits taken and earned 
during the school year. In addition, student internship participation during the year and 
responses to selected survey questions related to engagement (having led an activity, helped to 
plan an activity, contributed solutions to a community problem) were included. 
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These findings confirm the relationship between student level of assets and academic 
progress (credits), and the relationship between assets and student Constructive Use of 
Time—a domain that can be directly impacted by the GSS transfer school experience. It 
was expected, and was later the case, that the pre-enrollment DAP scores would have the largest 
predictive value. However, students who responded on the end-of-year survey that they had 
helped to solve a community problem also was identified as a predictor, as were students’ 
external and internal assets scale scores and the Constructive Use of Time subscale score. The 
number of credits earned during the school year was another predictor of the total DAP score, 
and internal assets, external assets, and Constructive Use of Time scale scores.>2 


Table 1 identifies the variables that were included in the predictive models, as well as those 
variables that were statistically significant predictors of the spring 2013 DAP scores. 


Table |. DAP regression analysis results 


External Internal School 
Total Asset Asset Context Constructive 
Scale Scale Scale Scale Use of Time 
Score Score Score Score Scale Score 
Variable (r2=0.546) (r2=0.433) (r2=0.528) (r?=0.304) (r2=0.573) 
Fall 2012 DAP score v v v v v 
Grade level in 2013 
Sex 
Black vs. other races v 


Hispanic vs. other races 


English language learner status in 
2013 


Eligibility for free/reduced price 
lunch in 2013 


Special education status in 2013 


Average daily attendance in 2013 


Credits taken in 2013 


Credits earned in 2013 v v v 


Age at start of SY 2012-13 


Had an internship during SY 2012- 
13 


52 See Appendix G, Tables G.1—G.5 for detailed results of the linear regression models. 
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External Internal School 
Total Asset Asset Context Constructive 
Scale Scale Scale Scale Use of Time 


Score Score Score Score Scale Score 
Variable (r2=0.546) (r2=0.433) (r2=0.528) (r2=0.304) (r2=0.573) 


Student Survey items (Spring 2013) 


Q3a. | led an activity (discussion 
group, service project) 

Q3c. | helped plan a program 
activity or event 

Q3zg. | contributed solutions to a if / 7 ff 


community problem 
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Conclusion and Recommendations 


The evaluation of the GSS transfer school model was designed to determine the extent to which 
the model is implemented at South Brooklyn and West Brooklyn Community High Schools and 
to determine, through a rigorous evaluation, the impact of the model on students’ academic 
performance and behavior. 


Implementation of the Model 


The GSS transfer school model incorporates those programmatic approaches that are believed 
to be effective in engaging students and, thus, in reducing dropout rates. All of the core 
components of the model were evident in the two GSS transfer schools. 


Information about implementation of comparable strategies in the other transfer schools from 
which comparison students were drawn is incomplete. All of the schools are small and have 
small class sizes, and all but one of them has a community organization on-site, with advocate 
counselors to play a similar role as the GSS staff. However, the other schools vary in the extent 
to which decision-making is shared between NYC DOE and community organization staff. 
Information was not available to assess the relationship between DOE and community 
organization leaders and staff. An obvious difference is that the GSS schools are neighborhood- 
based. 


Youth development practices were evident, but staff survey results identified a need to reinforce 
the staffs understanding of the schools’ youth development model. Although more than half of 
the staff reported that their school did a good or excellent job of orienting staff to the youth 
development model and providing professional development and training, more than a third did 
not share this view. Furthermore, staff's ratings of these aspects of the schools were lower in 
2013 than the year before. 


Youth Development Outcomes 


The absence of a comparison group for the assessment of youth development limits our ability 
to assess the impact of the GSS model on changes in these areas. Nevertheless, the DAP analysis 
findings confirm the relationship between assets and academic performance, as well as 
participation in extracurricular activities and civic engagement. The results also identified some 
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areas that warrant further exploration, including whether the program model is doing enough in 
terms of providing socio-emotional supports to students (and to their families) in order to 
bolster students’ internal assets. 


Recognizing that the final year of the evaluation was a time of substantial disruption and 
curtailment of school activities because of Superstorm Sandy, what can be made of these 
findings? They speak to the importance of consistently engaging students in learning and 
emphasizing the future focus of their education. The schools’ connections to their communities 
and the students’ desires to help solve community problems could be harnessed toward this goal. 


Impact of the Model 


Results from the outcome evaluation indicate that compared to matched groups of students 
from other transfer schools, the GSS transfer school model has a significant impact on 
graduation, school attendance, and credit attainment. The probability of GSS students graduating 
(63%) exceeds that of the comparison group by 12 percentage points. The rate for GSS students 
also is higher than the six-year graduation rate (Class of 2012) for over-age and under-credited 
(OA-UC) students in transfer schools citywide (41%) and far exceeds the rate for OA-UC 
students in traditional high schools (29%) (NYC DOE, 2014). 


The most notable findings of the exploratory subgroup analyses were that female GSS students 
and students who are admitted with less than 11 credits (and who thus have the longest path to 
graduation) have significantly higher probabilities of graduating than the comparison students. 


Other significantly positive findings on school attendance and credit attainment were found for 
the subgroups of females, males, Hispanic males, who represent the majority of the GSS 
students, and those with less than 11 credits at admission. 


Recommendations 


Based on the study findings, we offer the following recommendations for strengthening the 
model: 

1. Engage in further professional staff development about the GSS transfer school model 
as it relates to and incorporates youth development to ensure that staff are fully oriented 
to and comfortable with the model. 

2. Review strategies used to support students and whether there are additional ways to 
support their families through other GSS services. 

3. To capitalize on the positive findings associated with student engagement (ie., 
Constructive Use of Time), review the extracurricular and civic engagement 
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opportunities available to students with an eye toward expanding student interest in 
these areas and helping them to see the connections to academic performance and future 


well-being. 


The following recommendations are offered for further research: 

4. Conduct an impact study with additional cohorts to build the evidence base for the 
impact of the GSS transfer school model. 

5. Conduct a qualitative study of female students and students who entered the GSS 
schools with less than 11 credits to understand better why the GSS model produced 
such strong effect with these two groups. 

6. Explore the impact of the GSS transfer school model on students’ socio-emotional 
learning and civic engagement using a comparison group evaluation design. 
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Appendix A: Evaluation Design and Methods 


Guided by the What Works Clearinghouse (WWC) Procedures and Standards Handbook (v 2.1, 2011), 
Metis proposed conducting a rigorous and systematic evaluation that included formative and 
summative components and used multiple sources of quantitative and qualitative data and 
methods of analysis. The outcome evaluation assessed the extent to which the GSS model as 
implemented was meeting its objectives, and estimated the intervention impact on intended 
student outcomes based on treatment-comparison contrasts. The implementation evaluation was 
intended to provide formative feedback to the GSS model to improve implementation, and to 
examine the extent to which implementation occurred as planned. Findings from the 
implementation study were expected to provide contextual information for interpreting the 
impact study results (e.g., how variations in the model implementation may relate to intervention 
effectiveness). 


Evaluation of Outcomes 


In the evaluation of the GSS program models at South Brooklyn Community High School and 
West Brooklyn Community High School, all of the data needed for assessing student academic 
and behavioral outcomes were obtained from archival school records. With the generous 
permission of the NYC DOE, Metis was able to accumulate historical individual student level 
data records, including demographics, attendance and achievement metrics. Using these data 
Metis was able to create a retrospective longitudinal student-level file, covering the years prior to 
transfer school enrollment through the most recent years. The student-level baseline 
characteristics and target outcome data were therefore available for use in the current GSS 
impact study. 


The outcome evaluation design was intended to determine whether the GSS model had 
demonstrable impact on the academic and behavioral outcomes of their participants compared 
with a similar group of non-participants (i.e., counterfactual). Given the fact that a randomized 
controlled trial (RCT) design would not be feasible for the evaluation of the GSS program 
models, in accordance with the WWC guidelines, Metis proposed a rigorous quasi-experimental 
matched comparison group design based on a propensity score matching (PSM) approach. PSM 
is currently considered the best available approach to generating a comparable group of non- 
participants without random assignment. Under the PSM framework (Rosenbaum & Rubin, 
1983, 1984, 1985; Rosenbaum, 1991, 2002), initial large imbalances on observed covariates (e.g., 
demographic variables and baseline achievement) between treated and comparison groups could 
be removed or greatly reduced. PSM techniques first summarize all pertinent characteristics 
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observed prior to treatment (i.e., the matching variables) into a single score (i.e., the propensity) 
that indicates the predicted conditional probability of an individual participating in a given 
program. After propensity score estimation, PSM techniques typically match each program 
participant with one or more comparison students with similar propensity scores. One 
commonly used PSM technique for program evaluation is the nearest neighbor matching within 
caliper (also known as greedy matching). 


Metis adopted greedy matching without replacement algorithms in the outcome evaluation of the 
GSS model. The target students who were enrolled in the two GSS schools during the 2011-12 
school year were matched one-to-one to similar students in the remaining 11 transfer high 
schools in Brooklyn that do not follow the GSS model. The baseline variables used for 
propensity score estimation and matching included student gender, race/ethnicity, free/reduced 
price lunch (FRL) eligibility, English language learner (ELL) and special education status, New 
York State Grade 8 English Language Arts (ELA) and Math exam scores, number of years over- 
age when first enrolled in Grade 9, credits earned and average daily attendance (ADA) in Grade 
9. In addition, student grade level in the 2011-12 school year was used during the matching stage 
(ie., a 10% grade GSS student would be matched to a comparison 10% grader), although this 
variable was not involved in the propensity score estimation process since it was not baseline 
information. Following Rosenbaum and Rubin’s recommendation (1985), a caliper size of a 
quarter of standard deviation of the estimated propensity score was employed in the matching 
process. 


Note that propensity estimation and matching were done in two steps. Using the estimated 
propensity scores, matching was conducted multiple times with consideration given to the 
availability of pertinent outcomes for analysis. This was done to maximize the number of 
matched pairs with outcome data as well as ensure that the baseline equivalence of matching 
covariates could be established for the final analytic samples. Details on the level of overt bias 
reduction for each matching are presented in Tables A.1 through A.8 respectively for each final 
analytic sample.*3 As seen from these tables, while most matching variables displayed statistically 
significant differences between the target group and the potential comparison group before 
matching, these differences did not achieve statistical significance after matching. More 
importantly, the balance measure adopted by the WWC (ie., the standardized mean difference 
calculated by Hedge’s g) showed that for all the matching variables included in each PSM, the 
baseline equivalence was established after matching, either with or without the requirement for 
statistical adjustment of the baseline differences depending on the magnitude of Hedge’s g.*4 


53 For the graduation and persistence outcomes, matching was carried out twice for separate impact analyses of 
each at different time points—at the end of 2012-13 school year and at the end of 2013-14 school year (see 
Tables A.1, A.2, A.7, and A.8). 


4 Based on the WWC criteria, if the magnitude of Hedge’s g for a given baseline variable is (1) less than or 
equal to 0.05 standard deviations, one can conclude that equivalence is established for the baseline variable 
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In addition, each table indicates the matching rate for the overall target group based on GSS 
students who had complete data for matching and outcome analysis.*> ° According to Tables 
A.1 — A.8, all the matching rates were at least 90%, which means that the great majority of the 
target GSS youth successfully found comparable matches in the potential comparison group. 


(no statistical adjustment needed in outcome analyses later); (2) greater than .05 standard deviations but less 
than or equal to .25 standard deviations, one has to include the baseline variable in statistical models used in 
outcome analyses to account for the imbalance and establish baseline equivalence; and (3) greater than .25 
standard deviations, one has to conclude that equivalence was not established for the baseline variable (i.e., 
baseline imbalance). 


35 Group baseline equivalence must be demonstrated on the analysis sample that excludes cases with missing 
values because WWC guidelines do not allow missing data imputation for outcome or baseline matching 
variables when a study is based on a quasi-experimental design (QED). 


36 Note that there were a total of 429 GSS students who were enrolled in the two GSS schools during the 
2011-12 school year, but only 374 of them had complete matching data. 
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Table A.I. Baseline covariate balance before and after matching: GSS vs. 
comparison students for graduation vs. discharge status analysis (201 1-13) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 225 225 
Treated cases with complete 
‘ 248 

matching and outcome data 

Nereated lost after matching 23 

ereated lost after matching 9.3% 

Hispanic? 24.31" 67.65 71.11 69.78 -0.04 
Blacka 71.70" 14.71 14.22 14.22 0.00 
Wats cL en ee 3.99" 17.64" 14.67 16.00 0.06 

Fenalee 54.27* 47.86* 53.33 48.89 -0.11 
Males 45.73* 52.14* 46.67 51.11 0.11 

ELLa 2.02 * 4.28 3.56 4.89 0.20 
Not ELL 97.98 95.72" 96.44 95.11 -0.20 

FRLa 46.93 45.19 46.67 44.44 -0.05 
Not FRL@ 53.07 54.81 53.33 55.56 0.05 

Special Eda 8.58 9.63 5.78 7.56 0.17 
Not Special Ed? 91.42 90.37 94.22 92.44 -0.17 

637.16 * 640.99 * 641.08 638.23 
Grade 8 ELA -0.12 
(24.63) (23.53) (23.13) (24.29) 
638.60 ** bid 644.04 641.72 
Grade 8 Math ose -0.09 
(29.83) (24.94) (26.50) (24.46) 
Years of Over-Age When First 0.57" 0.34" 0.32 0.33 0.02 
Enrolled in Grade 9¢ (0.70) (0.52) (0.53) (0.52) : 
6.98 6.88 742 6.78 
Grade 9 Credits Earned -0.15 
(4.32) (4.34) (4.11) (4.36) 
80.07 * 82.40* 82.57 81.99 
Grade 9 ADAs -0.04 
(16.83) (15.93) (14.61) (16.55) 


* For the categorical matching variables, column percentage for each group is presented. 

> Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.2. Baseline covariate balance before and after matching: GSS vs. 
comparison students for enrollment vs. discharge status analysis (201 1-13) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 200 200 
Treated cases with complete 
‘ 218 

matching and outcome data 

Nereated lost after matching 18 

ereated lost after matching 8.3% 

Hispanic? 24.31" 67.65 68.50 72.00 0.10 
Blacka 71.70" 14.71 16.50 16.50 0.00 
Wats cL en ee 3.99" 17.64" 15.00 11.50 -0.19 

Fenalee 54.27* 47.86* 53.00 50.50 -0.06 
Males 45.73* 52.14* 47.00 49.50 0.06 

ELLa 2.02 * 4.28 4.50 4.50 0.00 
Not ELL 97.98 95.72" 95.50 95.50 0.00 

FRLa 46.93 45.19 47.50 46.00 -0.04 
Not FRL@ 53.07 54.81 52.50 54.00 0.04 

Special Eda 8.58 9.63 9.00 9.50 0.04 
Not Special Ed? 91.42 90.37 91.00 90.50 -0.04 

637.16 * 640.99 * 639.72 640.82 
Grade 8 ELA 0.05 
(24.63) (23.53) (21.73) (20.18) 
638.60 ** bid 643.89 643.80 
Grade 8 Math ose 0.00 
(29.83) (24.94) (24.04) (22.64) 
Years of Over-Age When First 0.57" 0.34" 0.40 0.35 0.09 
Enrolled in Grade 9¢ (0.70) (0.52) (0.58) (0.52) : 
6.98 6.88 6.37 6.73 
Grade 9 Credits Earned 0.08 
(4.32) (4.34) (4.10) (4.50) 
80.07 * 82.40* 79.40 80.51 
Grade 9 ADAs 0.07 
(16.83) (15.93) (16.52) (16.26) 


* For the categorical matching variables, column percentage for each group is presented. 

» Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.3. Baseline covariate balance before and after matching: GSS vs. 
comparison students for credits earned analysis (201 1-13) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 348 348 
Treated cases with complete 
F 374 

matching and outcome data 

Nereated lost after matching 26 

ereated lost after matching 7.0% 

Hispanic? 24.31" 67.65 70.11 70.69 0.02 
Blacka 71.70" 14.71 15.80 15.80 0.00 
Wats cL en ee 3.99" 17.64" 14.08 13.51 -0.03 

Fenalee 54.27* 47.86* 52.87 49.7| -0.08 
Males 45.73* 52.14* 47.13 50.29 0.08 

ELLa 2.02 * 4.28 4.02 4.02 0.00 
Not ELL 97.98 95.72" 95.98 95.98 0.00 

FRLa 46.93 45.19 47.70 45.98 -0.04 
Not FRL@ 53.07 54.81 52.30 54.02 0.04 

Special Eda 8.58 9.63 8.62 8.91 0.02 
Not Special Ed? 91.42 90.37 91.38 91.09 -0.02 

637.16 * 640.99 * 640.29 639.73 
Grade 8 ELA -0.02 
(24.63) (23.53) (22.17) (22.65) 
638.60 ** bid 643.65 643.11 
Grade 8 Math ose -0.02 
(29.83) (24.94) (25.38) (24.43) 
Years of Over-Age When First 0.57" 0.34" 0.37 0.36 0.02 
Enrolled in Grade 9¢ (0.70) (0.52) (0.56) (0.53) : 
6.98 6.88 7.20 6.86 
Grade 9 Credits Earned -0.08 
(4.32) (4.34) (4.21) (4.36) 
80.07 * 82.40* 81.95 81.88 
Grade 9 ADAs 0.00 
(16.83) (15.93) (15.19) (16.07) 


* For the categorical matching variables, column percentage for each group is presented. 

» Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.4. Baseline covariate balance before and after matching: GSS vs. 
comparison students for number of core Regents exams passed analysis (201 1-13) 


GSS vs. Comparison Students 


Matching Variable 


Before Matching After Matching 


Comparison Comparison Hedge’s g 

Count 2,180 374 99 99 

Treated cases with complete 

‘ 110 

matching and outcome data 

Nereated lost after matching II 

ereated lost after matching 10.0% 

Hispanic: 24.31" 67.65 ** 70.71 68.69 -0.06 
Black 71.70** 14.71 19.19 19.19 0.00 
andthe 3.99" 17.64 10.10 pay: 0.12 

Females 54.27* 47.86 * 60.61 54.55 -0.15 
Males 45.73* 52.14* 39.39 45.45 0.15 

ELLa 2.02* 4.28 3.03 4.04 0.18 
Not ELL 97.98 * 95.72" 96.97 95.96 -0.18 

FRLa 46.93 45.19 4747 50.51 0.07 
Not FRL? 53.07 54.81 52.53 49.49 -0.07 

Special Eda 8.58 9.63 V1.1 13.13 0.11 
Not Special Eda 91.42 90.37 88.89 86.87 -0.1 1 

637.16 ™ 640.99 * 640.38 642.75 
Grade 8 ELAs 0.13 
(24.63) (23.53) (19.04) (18.70) 
638.60 ** i 647.91 645.96 
Grade 8 Math lieu -0.09 
(29.83) (24.94) (19.31) (21.58) 
Years of Over-Age When First 0.57" 0.34 ™ 0.39 0.38 0.02 
Enrolled in Grade 9¢ (0.70) (0.52) (0.60) (0.53) , 
6.98 6.88 7.18 7.23 
Grade 9 Credits Earned* 0.01 
(4.32) (4.34) (4.16) (4.46) 
80.07 * 82.40* 82.65 82.82 
Grade 9 ADA< 0.01 
(16.83) (15.93) (11.95) (14.22) 


* For the categorical matching variables, column percentage for each group is presented. 
» Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 


° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 


the parentheses. 


™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.5. Baseline covariate balance before and after matching: GSS vs. 
comparison students for school attendance analysis (201 1-13) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 347 347 
Treated cases with complete 
F 374 

matching and outcome data 

Nereated lost after matching 27 

ereated lost after matching 7.2% 

Hispanic? 24.31" 67.65 70.32 70.89 0.02 
Blacka 71.70" 14.71 15.85 15.85 0.00 
Wats cL en ee 3.99" 17.64" 13.83 13.26 -0.03 

Fenalee 54.27* 47.86* 53.60 49.86 -0.09 
Males 45.73* 52.14* 46.40 50.14 0.09 

ELLa 2.02 * 4.28 4.03 4.03 0.00 
Not ELL 97.98 95.72" 95.97 95.97 0.00 

FRLa 46.93 45.19 47.84 46.11 -0.04 
Not FRL@ 53.07 54.81 52.16 53.89 0.04 

Special Eda 8.58 9.63 8.36 8.93 0.04 
Not Special Ed? 91.42 90.37 91.64 91.07 -0.04 

637.16 * 640.99 * 640.38 639.67 
Grade 8 ELA -0.03 
(24.63) (23.53) (22.25) (22.66) 
638.60 ** bid 643.68 643.12 
Grade 8 Math ose -0.02 
(29.83) (24.94) (25.37) (24.46) 
Years of Over-Age When First 0.57" 0.34" 0.37 0.36 0.02 
Enrolled in Grade 9¢ (0.70) (0.52) (0.56) (0.53) : 
6.98 6.88 7.20 6.85 
Grade 9 Credits Earned -0.08 
(4.32) (4.34) (4.23) (4.37) 
80.07 * 82.40* 81.92 81.84 
Grade 9 ADAs -0.01 
(16.83) (15.93) (15.18) (16.07) 


* For the categorical matching variables, column percentage for each group is presented. 

» Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.6. Baseline covariate balance before and after matching: GSS vs. 
comparison students for suspensions analysis (201 l-13) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 348 348 
Treated cases with complete 
F 374 

matching and outcome data 

Nereated lost after matching 26 

ereated lost after matching 7.0% 

Hispanic? 24.31" 67.65 70.11 70.69 0.02 
Blacka 71.70" 14.71 15.80 15.80 0.00 
Wats cL en ee 3.99" 17.64" 14.08 13.51 -0.03 

Fenalee 54.27* 47.86* 52.87 49.7| -0.08 
Males 45.73* 52.14* 47.13 50.29 0.08 

ELLa 2.02 * 4.28 4.02 4.02 0.00 
Not ELL 97.98 95.72" 95.98 95.98 0.00 

FRLa 46.93 45.19 47.70 45.98 -0.04 
Not FRL@ 53.07 54.81 52.30 54.02 0.04 

Special Eda 8.58 9.63 8.62 8.91 0.02 
Not Special Ed? 91.42 90.37 91.38 91.09 -0.02 

637.16 * 640.99 * 640.29 639.73 
Grade 8 ELA -0.02 
(24.63) (23.53) (22.17) (22.65) 
638.60 ** bid 643.65 643.11 
Grade 8 Math ose -0.02 
(29.83) (24.94) (25.38) (24.43) 
Years of Over-Age When First 0.57" 0.34" 0.37 0.36 0.02 
Enrolled in Grade 9¢ (0.70) (0.52) (0.56) (0.53) : 
6.98 6.88 7.20 6.86 
Grade 9 Credits Earned -0.08 
(4.32) (4.34) (4.21) (4.36) 
80.07 * 82.40* 81.95 81.88 
Grade 9 ADAc 0.00 
(16.83) (15.93) (15.19) (16.07) 


* For the categorical matching variables, column percentage for each group is presented. 

» Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.7. Baseline covariate balance before and after matching: GSS vs. 
comparison students for graduation vs. discharge status analysis (201 1-14) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 291 291 
Treated cases with complete 
F 314 

matching and outcome data 

Nereated lost after matching 23 

ereated lost after matching 7.3% 

Hispanic? 24.31" 67.65 72.51 71.48 -0.03 
Blacka 71.70" 14.71 14.78 14.78 0.00 
Wats cL en ee 3.99" 17.64" 12.71 13.75 0.05 

Fenalee 54.27* 47.86 * 53.61 48.80 -0.12 
Males 45.73* 52.14* 46.39 51.20 0.12 

ELLa 2.02 * 4.28" 4.12 481 0.10 
Not ELL 97.98 95.72" 95.88 95.19 -0.10 

FRLa 46.93 45.19 46.74 43.64 -0.08 
Not FRL@ 53.07 54.81 53.26 56.36 0.08 

Special Eda 8.58 9.63 9.62 8.25 -0.10 
Not Special Ed? 91.42 90.37 90.38 91.75 0.10 

637.16 * 640.99 * 640.06 640.53 
Grade 8 ELA 0.02 
(24.63) (23.53) (22.66) (22.66) 
638.60 ** bid 642.77 643.81 
Grade 8 Math ose 0.04 
(29.83) (24.94) (26.57) (24.33) 
Years of Over-Age When First 0.57" 0.34" 0.32 0.35 0.06 
Enrolled in Grade 9¢ (0.70) (0.52) (0.54) (0.53) : 
6.98 6.88 6.93 6.76 
Grade 9 Credits Earned -0.04 
(4.32) (4.34) (4.01) (4.27) 
80.07 * 82.40* 82.62 82.26 
Grade 9 ADAs -0.02 
(16.83) (15.93) (13.85) (16.04) 


* For the categorical matching variables, column percentage for each group is presented. 

> Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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Table A.8. Baseline covariate balance before and after matching: GSS vs. 
comparison students for enrollment vs. discharge status analysis (201 I-14) 


GSS vs. Comparison Students 


Matching Variable Before Matching After Matching 
Comparison Comparison Hedge’s g 
Count 2,180 374 155 155 
Treated cases with complete 
‘ 168 

matching and outcome data 

Nereated lost after matching 13 

ereated lost after matching 7.7% 

Hispanic? 24.31" 67.65 68.39 72.26 0.11 
Blacka 71.70" 14.71 15.48 15.48 0.00 
Wats cL en ee 3.99" 17.64" 16.13 12.26 -0.19 

Fenalee 54.27* 47.86 * 50.32 49.03 -0.03 
Males 45.73* 52.14* 49.68 50.97 0.03 

ELLa 2.02 * 4.28" 3.87 4.52 0.10 
Not ELL 97.98 95.72" 96.13 95.48 -0.10 

FRLa 46.93 45.19 50.32 48.39 -0.05 
Not FRL@ 53.07 54.81 49.68 51.61 0.05 

Special Eda 8.58 9.63 7.10 8.39 0.11 
Not Special Ed? 91.42 90.37 92.90 91.61 -0.11 

637.16 * 640.99 * 637.14 639.86 
Grade 8 ELA 0.11 
(24.63) (23.53) (27.50) (20.82) 
638.60 ** bid 643.67 642.28 
Grade 8 Math ose -0.06 
(29.83) (24.94) (23.05) (22.62) 
Years of Over-Age When First 0.57" 0.34" 0.37 0.35 0.04 
Enrolled in Grade 9¢ (0.70) (0.52) (0.51) (0.50) : 
6.98 6.88 6.29 6.57 
Grade 9 Credits Earned 0.06 
(4.32) (4.34) (4.14) (4.73) 
80.07 * 82.40* 80.48 79.34 
Grade 9 ADAs -0.07 
(16.83) (15.93) (15.36) (17.43) 


* For the categorical matching variables, column percentage for each group is presented. 

> Other include those who were Asian, Native American/Alaskan, Native Hawaiian/Pacific Islander, or multiracial. 

° For the continuous matching variables, group mean is presented first, followed by the corresponding standard deviation in 
the parentheses. 

™ < 001, ** < .01, * < .05, chi-square test or independent-samples t-test two-tailed. 
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After generating a closely matched group of comparison students for the GSS students, 
hierarchical linear modeling (HLM) was used to assess the overall impact of the GSS model on 
students’ academic and behavioral outcomes. Additional exploratory analyses were also 
conducted for the following subgroups: (1) males, (2) females, (3) Black males, (4) Hispanic 
males, and (5) students with less than 11 credits at admission. Because school was the unit of 
assignment but impacts were measured at the student level, two-level models were used for these 
cross-sectional analyses to account for the clustering of students within schools. In addition to 
the treatment indicator, the matching variables used for PSM were also included as pertinent 
covariates in the student-level models of HLMs to further control for possible confounding 
effects in the impact analyses. In addition, Metis tried to include a few important school 
characteristics (i.e., percent low-income and average New York State Grade 8 ELA exam score) 
in the school-level models of HLMs for more precise estimates of intervention impacts. The 
limited number of schools (2 GSS and 11 comparisons) posed challenges to including additional 
school-level variables in HLMs. In some cases, the between-school variance components in the 
HLM models could not converge, and therefore the corresponding analyses were re-conducted 
using regular regressions (i.¢., without nesting students under schools). Additional details on the 
regression analyses are presented in Appendices D and E. 


Evaluation of Implementation 


Information about implementation at the GSS schools was obtained from interviews, focus 
groups, surveys, school walk-throughs, and from documentation and data provided by GSS and 
the NYC DOE. 

e Site visits were conducted in the fall of each school year for the purpose of conducting 
individual interviews with the GSS and DOE administrators, focus groups with students 
and staff, and school walk-throughs to get a feel of the school climate. 

e Students were surveyed in December 2011, spring 2012, and spring 2013. 

e Staff were surveyed in spring 2012 and spring 2013. 

e Data on the characteristics of the students enrolled in the GSS schools were obtained 
from the NYC DOE. 

e Rosters of enrolled students and data on the students’ participation in internships were 


provided by GSS. 


Information on practices in the comparison group transfer schools was obtained from interviews 
with some of the representatives of the community-based organizations partnering with or 
providing services to students in these schools as well as from information available to the public 
in the NYC DOE high school directory and on the department’s website. 
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Protocols were designed to collect information about the key components of the GSS transfer 
school model, respondents’ experiences in the schools and services provided, challenges, and 
recommendations for improvement. 


Methods and Sources of Data 


In each year of the evaluation individual interviews were conducted with each school principal 
and GSS program director. Two staff focus groups were conducted at each school each year, 
with participation from a total of 29 DOE and GSS personnel across both schools. Two student 
focus groups were conducted at each school each year, with a total of 29 students participating in 
a focus group in the first year and 23 in the second year. Half of the groups were composed of 
student leaders while the other half was selected by advocate counselors from across their 
caseloads based on their ability to express themselves in a group. 


A small number of parents participated in a focus group at each school; nine parents participated 
the first year and six the second year. Focus groups were conducted in Spanish for Spanish- 
speaking parents. 


The evaluation team took a tour of each school led by student or school leaders. The walk- 
through included observations of classrooms (although not of instruction); common spaces, 
such as the cafeteria, stairwells, and hallways; and rooms used for counseling. The purpose of the 
tours was to become familiar with the physical facilities and get a sense of the school atmosphere 
that could be observed through wall posters, exhibits of student work, etc. Instruction was 
minimally observed during the school visits. 


A locally-developed survey was administered during the school day by school personnel in 
December 2011, May 2012, and May 2013, according to the schedule listed above. The survey 
was designed to gather information about students’ educational experiences, opportunities to 
participate in community-building and leadership experiences, relationships with peers and adults 
at the school, experiences with college- and career-preparation activities, educational aspirations, 
and satisfaction. It included adapted standardized items focused on key predictors of 
engagement as well as locally developed items that addressed specific aspects of the GSS transfer 
school model. All enrolled students were expected to be surveyed. Surveys were administered, in 
paper versions, primarily by the advocate counselors in a group setting. Using rosters provided 
by GSS, the survey was pre-populated with students’ names, but in addition, the schools received 
copies with names left blank for any new students. 


An online survey was administered to all school staff over a two-week period in May 2012 and 
May 2013. The survey was designed to obtain staff perspectives on how well their school is 


implementing the model’s core principals and components. 


Student and staff response rates for each year’s administration are presented in Table A.7. 
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Table A.7. Survey response rates 


TotalN ResponseN Response %  TotalN Response N | Response % 


Spring 2012 352 280 79.6% 64 6l 95.3% 


Spring 2013 327 239 73.1% 56 53 94.6% 


Development of the surveys and protocols was informed by reports by and about the GSS 
program, including a manual for replicating the model, a logic model, and the Good Shepherd 
Services Sanctuary Information Guide. Student rosters and program administrative data 
(assignment of advocate counselors, participation in internships) also were obtained from GSS. 


To identify transfer schools for the comparison group, the list of all transfer schools in Brooklyn 
was teviewed by GSS staff. Schools affiliated with or that follow the GSS model were eliminated 
from the list, leaving a total of 11 schools. Descriptions of the schools and information about 
enrollment were obtained from the NYC DOE website and the directory of high schools. A 
semi-structured telephone interview protocol was developed and, beginning in January 2013, 
outreach was made by Metis researchers to the principals of these schools to obtain qualitative 
information about the schools. Two schools indicated they did not wish to participate, and the 
others did not return emails and telephone calls. After repeated attempts, the same methods 
were used to contact the program directors of the community-based organizations that work in 
partnership and/or provide on-site services. Outreach was also made by GSS staff and staff of a 
coalition that works with these schools. The evaluators conducted interviews or received written 
responses from the program directors of community organizations that work in five of the 11 
schools. 
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Appendix B: Traditional High School Comparison 
Group Considerations 


In the evaluation of the GSS program, Metis initially tried to limit the potential comparison 
schools to those with similar core educational programs and geographical proximity to the target 
schools (i.e., the other Brooklyn transfer high schools that do not follow the GSS model) to 
better control for unmeasured confounding variables. However, instead of relying solely on a 
comparison group from other transfer schools offering similar programs, GSS asked Metis to 
explore the possibility of generating an additional matched comparison group from similar over- 
age, under-credited students who stayed in the sending regular high schools using propensity 
score matching (PSM) techniques. 


Although aware that there might be difficulty in finding similar students from the regular high 
schools, Metis continued with the design as planned. A retrospective longitudinal student-level 
file was generated from NYC DOE data for students in the identified sending high schools, with 
the same matching metrics that were used for creating the transfer school comparison group.*” 
Again using the same greedy matching w7thout replacement algorithms, the GSS target students 
enrolled in the two schools during the 2011—12 school year were matched to similar youth who 
stayed in the 10 major sending high schools. 


While target case losses after PSM are very common due to stringent matching criteria as well as 
missing or faulty data, there is an expectation that these losses can be minimized with a large 
reservoir of non-participating subjects and adequate overlap between target and possible 
comparison samples. The PSM matching process as designed for this study involved a large set 
of matching variables for a relatively small group of targets (N= 429).38 It is always preferable to 
have a large pool of possible comparisons to minimize loss, as explained above. The matching 
procedures for both possible comparison groups were repeated several times to ensure a 


37 Recall that the following matching variables were used for identifying the transfer school comparison group: 
gender, race/ethnicity, free/reduced price lunch (FRL) eligibility, English language learner (ELL) and special 
education status, New York State Grade 8 English Language Arts (ELA) and Math exam scores, number of 
yeats over-age when first enrolled in Grade 9, credits earned and average daily attendance (ADA) in Grade 9. 


38 Note that while there were a total of 429 GSS students who were enrolled in the two schools during the 
2011-12 school year, only 374 of them had complete matching data, and among these 374 cases, some were 
further missing various target outcomes. 
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minimal loss of cases.3? However, even though the traditional high school comparison pool 
(N=30,138) was over ten times as large as the available transfer school comparison pool 
(N=2,581), extensive matching and re-matching only accounted for eight (8) more matches for 
the traditional high school than the much smaller transfer school population. This was a clear 
indication that there may not be a great deal of overlap between the transfer school target 
population and the population of students who remain in the sending high schools. 


Nonetheless, with a traditional high school comparison group identified, Metis proceeded to 
conduct the impact analyses for the following outcomes: (1) graduation vs. discharge status, (2) 
enrollment vs. discharge status, (3) credits earned, (4) number of core Regents exams passed, (5) 
attendance, and (6) suspensions. Table B.1 presents a summary of the regression analyses results. 
The findings indicated that GSS students had a significant /ower probability of remaining enrolled 
in school (as opposed to being discharged) than their matched comparisons from the top ten 
traditional feeder high schools (odds ratio = 0.441, p-value = 0.0326). Observed differences for 
the other metrics proved to be negligible (graduation), positive (credits earned) or negative (core 
Regents passed, average daily attendance (ADA), suspensions) and not statistically significant 
with, at best, very small effect sizes (see Table B.1). 


In attempting to interpret the counter-intuitive results, Metis and the GSS program staff revisited 
the matching variables that were used for selecting similar comparison students. Following the 
What Works Clearinghouse (WWC) Procedures and Standards Handbook (v 2.1, 2011), Metis tried to 
demonstrate baseline equivalence of the target GSS group and the traditional high school 
comparison group on student race/ethnicity, gender, measures of degree of disadvantage, and 
school performance. Based on the feedback from GSS, however, in the case of traditional high 
school comparison selection, the credits earned and ADA in grade 9 were actually not good 
proxies for prior high school performance, which is crucial for identifying those who were truly 
comparable to the target youth. The key issue is that these variables are measured in Year 1 of 
high school and do not provide accurate information for important baseline characteristics that 
GSS and other transfer schools use to select their students: being 16 or older, over-age, under- 
credited, and truant at the time of enrollment.*” In other words, the selected ninth-grade metrics 
therefore do not provide an adequate account of student achievement and/or attendance prior 
to transfer school entry for comparison students selected from the traditional high schools. 
Furthermore, given that only those who stayed in sending schools avd had pertinent outcomes 
could be matched to the target students, it was impossible to find the truant students who would 
likely be a better comparison population in the traditional high schools. 


» An iterative matching process is typically more robust given that greedy matching pairs treated subjects with 
their closest comparison counterparts one at a time without reconsidering early matches as later matches are 
made. 


4 Note that approximately 45% of the target youth enroll in the GSS schools when they are 16, while the 
remaining 55% are 17 to 20 years old at the time of enrollment. 
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Given the difficulty Metis encountered in collecting a matched comparison group from over 
30,000 possible comparisons and the rolling admission policy used by the transfer schools, the 
task of constituting a closely-matched comparison group based on mote accurate baseline 
characteristics seemed insurmountable. Metis and GSS thus concluded that any comparison 
group from traditional feeder high schools would likely not accurately reflect the treatment 
group in baseline equivalence, particularly if based on the above selected matching variables. 
According to the WWC standards, this poses an insurmountable threat to the internal validity of 
the comparison, leading to the conclusion that the comparison to traditional feeder schools 
presented in this appendix is inherently flawed and should not be included in the text of the final 
report. The comparisons presented within this appendix are therefore provided for disclosure 
purposes only and should neither be presented nor interpreted as a valid statistical comparison 
of similarly situated groups. 


While the comparison students from the other Brooklyn transfer schools were selected using the 
same matching variables to denote prior high school performance, the matching variables are 
more appropriate for this group of students. The potential comparison group was restricted to 
those who had already been admitted to transfer schools with similar admission criteria to the 
GSS schools. Because we know that these students are like those in the treatment group based 
on selection criteria to the transfer schools, the use of the ninth-grade metrics to approximate 
baseline performance is appropriate and not a threat to internal validity. The impact analyses 
based on the transfer school comparison therefore could meet the WWC evidence standards 
with reservations, since the baseline equivalence of both groups was established for final analytic 
samples based on valid observed pre-intervention characteristics related to the outcomes of 
interest (see Appendix A for details). 
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Table B.|. Summary of regression results for overall impacts on traditional high 


school student comparison group (201 1-13)4! 


D egre 0 ad e e Odd 
LD) otomo ian " Oda ari 
ed D e D e 
o Sue erse of Odd 
Comparison Treatment Comparison Treatment 
Graduation vs. = 
226 x2 1.690 LSI 1.698 1.602 0.943! = 1.060 0.8771 
Discharge 
Se lene NS, 206 x 2 2.492 1.191 2.763 1218 | 0.441-1=2.268 0.0326 
Discharge 
ue nadisred Means Regression-Adjusted e 
O O Means xy = 
O e) e He se D e 
Comparison Treatment Comparison Treatment De 
Credits Earned 346 x2 12.428 12.822 12.449 12818 0.369 0.038 0.7303 
SING 50) 104x2 2.529 2.260 2.555 2.234 = 0.321 -0.179 | 0.1187 
Passed 
Average Daily 
Attendance SY12 356x2 66.838 64.896 66.950 64.783 -2.168  -0.074 0.6620 
& SYI3 
Bee one 356 x2 0.126 0.076 0.125 0.084 —-0.041 _— -0.076 0.3699 


“1 Note that the second round of analyses of the graduation and persistence outcomes at the end of 2013-14 
school year was not carried out for the traditional high school group. 
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Appendix C: The GSS Model Compared to 
Comparison Group Schools 


Information collected through interviews and available documentation indicates that, on the 
whole, the GSS transfer schools have many aspects in common with the transfer schools from 
which comparison students were drawn. However, there was a range of practices at these other 
schools and information about these schools was limited to public sources and interviews with 
representatives of the some of the community organizations. Highlights of the similarities and 
differences are presented below. 


Admissions criteria and practices: Unlike the GSS schools, the comparison schools do 
not serve a neighborhood catchment area. Students in all transfer schools must have been 
enrolled in another high school for at least one year, however different schools have various 
credit entry requirements and truancy is not always a criterion. 


Based on interview information obtained from five of the comparison schools, the schools 
follow an admissions process that is similar to that of the GSS schools, including active 
recruitment and referrals from guidance counselors, one-on-one meetings of students and 
parents/guardians with counselors, reviews of transcripts to make sure students have sufficient 
time to be able to graduate, and reading (and at some schools math) assessments. With one 
exception, at the interviewed schools, the community organization staff handles the intake and 
admissions process, and enrollment decisions are made jointly by NYC DOE and community 
organization personnel. 


The transfer schools vary in when they admit students during the school year, with some 
offering three cycles, one having rolling admissions, and others following a semester schedule. 
Some schools maintain a waiting list. 


Leadership and partnership structure: The structures at the comparison schools vary, 
with some having shared leadership and others a more differentiated relationship. All but one of 
them has a community-based organization on-site;#? however, they vary in the extent to which 
decision making is shared between NYC DOE and community organization staff. Information 


* None of the other Brooklyn transfer schools has a partnership with GSS as this was a criterion for excluding 
schools from this comparison group. 
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was not available to assess the relationship between DOE and community organization leaders 
and staff. 


Integration of advocate counseling: The advocate counselor role is present at some, but 
not all, of the comparison schools. At some of them, various school staff have assumed some of 
these functions—for example, attendance outreach and academic advisement. Information from 
interviews indicates that two of the schools are implementing models that closely resemble the 
primary person model used by GSS. The other schools fall somewhere along the spectrum 
between a primary person model and the approach adopted by most traditional high schools in 
which attendance, academic, guidance, and career/college readiness supports are offered by 
different school personnel, and in only a few instances, by community organization staff. 


Personalized, small school environment: The GSS schools are similar to the comparison 
schools in size and student-to-staff ratios. The comparison schools have small student 
enrollments, ranging from 125 to 370, with a typical average class size of about 15 to 20 
students. According to interviews, the community organization partners have played a critical 
role in helping provide the personalized environment and types of supports that students need 
to succeed and graduate. 


Youth development approach to instruction: In terms of how students are grouped and 
an individualized approach, the transfer school model developed by GSS and adopted by DOE 
is present in the comparison schools. Like the GSS schools, each of the transfer schools is 
structured on credit needs rather than grade level. Information from the interviews indicates that 
two follow a semester schedule, two offer three academic cycles, and one is based on student 
mastery of academic content. Information was not available for the comparison schools on 
opportunities for youth voice and participation. 


Postsecondary preparation: Nine of the 11 transfer schools in the comparison group have 
a Learning to Work (LTW) program in their school. In these schools, the LTW director and/or 
coordinator and, in many cases additional staff from the partnering community organizations, 
are responsible for providing students with job and career development activities, internships, 
and college exploration activities. 


S) metis associates making a meaningful difference 72 


RE-ENGAGING YOUTH FOR HIGH SCHOOL SUCCESS 


Appendix D: Regression Analysis Results, 201 |—13 


Full model specifications and regression analysis results are presented below. Due to the 
multiple dimensions of the data structure in this study (ie., students nested within schools), 
hierarchical linear modeling (HLM) was conducted to account for the clustering effect and 
control for multiple covariates at each level within the same analysis. In cases where the 
between-school variance was zero (and therefore the corresponding random component 
estimate did not show up), standard regressions were used to re-analyze the same sets of data 
and provide the final results. Note that in each regression model, all covariates were grand-mean 
centered, except for the treatment indicator. 


Full Hierarchical Linear Regression Model 


Level 1: Student level 
Y,, = By; + A, (Male, — Male..)+ £, (Hispanic, — Hispanic. )+ f, (Black,; — Black..) 


+ B,,(SpeEd,, -SpeEd..)+ 8,,(FRL, -FRL.)+ £,,(ELL, —ELL..) 

+ 8, (GRD8ELA, -GRD8ELA.)+ £, (GRD8Math, -GRD8Math.) 
+ B, (OAGRD9Y, -OAGRD9.)+ £,,,(GRD9ICRD, —GRD9CRD.)+ 
+ B,,, (GRD9ADA, —GRD9ADA.)+1; 


where 


Y,, represents the selected outcome for student 7 in school /; 


y 
f,; tepresents the mean score for school / adjusted for the student-level covariates; 


B,,;— A; tepresent the regression coefficients for school /, associated with various 
student-level covariates; and 


r, teptesents the random error associated with student 7in school /. 


43 Specifically, in addition to the treatment indicator (IRT), the following covariates were included in each full 
model: student gender (Male), race/ethnicity (Hispanic, Black), free/reduced price lunch eligibility (FRL), 
English language learner (ELL) and special education status (SpeEd), New York State Grade 8 English 
Language Arts and Math exam scores (GRD8ELA, GRD8Math), number of years over-age when first 
enrolled in Grade 9 (OAGRDS), credits earned and average daily attendance (ADA) in Grade 9 (GRD9ICRD, 
GRD9IADA), and percent low-income and average New York State Grade 8 ELA exam score at the school- 
level (PFRL, AVGGRD8ELA). 
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Level 2: School level 
Bo; = Yoo + Yo (AVGGRDS8ELA , — AVGGRD8ELA.) + 7),(PFRL , — PFRL.) + 


+ Yo3(TRT , ) + Lp j 

Ay =Y10 
Boj = Y20 
fj = Y30 
Ba; = Y40 
sj = Y50 
Boj = Yeo 
fr = Y70 
Bsj = 780 
oj = Yoo 
P10; = 100 
Bij = Yi10 
where 


Yoo tepresents the selected outcome mean for the comparison schools; 
Yo-Yo teptesent the regression coefficients associated with the school-level covariates; 


Yo; tepresents the regression coefficient associated with the treatment indicator — it 


quantifies the treatment impact (the mean difference in the outcome between treatment 
and comparison subjects); 


Yio-—Viy tepresent the common regression coefficients associated with the various 
student-level covariates for each school; and 


My; tepresents the random error associated with school /. 


Full Hierarchical Generalized Linear Regression Model 


The two-level logistic regression model is given in terms of the logits of probabilities of the 
selected outcome equal to 1, Le., 


The full multilevel model can be specified as follows: 
Level 1: Student level 


1; = A; + a, ;(Male,, — Male..) + a, | (Hispanic, — Hispanic.) + a, (Black,; — Black..) 
+ a, (SpeEd,, —SpeEd..) + a, |(FRL,,— FRL..) + @ CELL, — ELL.) 
+a, ,(GRD8ELA,,— GRD8ELA..) + a ,(GRD8Math,, - GRD8Math..) 
+ a, (OAGRD9,, —- OAGRD9..) + a%,),(GRD9CRD ,, — GRD9CRD..) 
+a,,,(GRD9ADA,, — GRD9ADA..) + 17, 


where 
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Qj 


represents the mean logit for school / adjusted for the student-level covariates; 
a@,;—,,, tepresent the regression coefficients for school /, associated with various 
student-level covariates; and 
r, tepresents the random error associated with student / in school /. 
Level 2: School level 
O%); = Byy + By (AVGGRD8ELA , — AVGGRD8ELA.) + f,,.(PFRL ; — PFRL.) 
ae {,(TRT ;) + Up; 


a; = Bo 


a) = Bry 
a3, = Ba 
Ay; = By 
as; = By 
ej = Bo 
a= Br 
ay = Bao 
Qy j= Bon 
Qo; = Boo 
a1) = Bw 
Where 


Bo teptesents the mean logit for the comparison schools; 
By — Bor teptesent the regression coefficients associated with the school-level covariates; 
Bo, teptesents the regression coefficient associated with the treatment indicator — it 


quantifies the treatment impact (the difference in the log-odds-ratio associated with 
being a treatment subject, as opposed to a comparison subject); 

g ) J 
Bio Bry Teptesent the common regression coefficients associated with various student- 


level covariates for each school; and 


Uy; tepresents the random error associated with school /. 
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Full Linear Regression Model 

Y, = £, + B, (Male, — Male.) + 8, (Hispanic, — Hispanic.) + , (Black, — Black.) 
+ B,(SpeEd, — SpeEd.)+ 6; (FRL, — FRL.)+ £, (ELL, — ELL.) 
+ B, (GRD8ELA, — GRD8ELA.)+ 8, (GRD8Math, — GRD8Math.) 
+ B, (OAGRD9, — OAGRDS.) + f,,(GRD9ICRD, - GRD9CRD.) 
+ 8, (GRD9CRD, — GRD9CRD.)+ ,,(AVGGRD8ELA, — AVGGRD8ELA.) 
+ B,,(PFRL, —PFRL)+ £,,(TRT) +, 


where 


Y, represents the selected outcome for subject ¢ 
[By tepresents the mean score for subject 7 adjusted for the covariates; 


£, — £3 tepresent the regression coefficients associated with various covariates for 
subject Z 
f,, represents the regression coefficient associated with the treatment indicator — it 


quantifies the treatment impact (the mean difference in the outcome between treatment 
and comparison subjects); 


€, represents the random error associated with subject 7. 


Full Logistic Regression Model 


The logistic regression model is given in terms of the logits of probabilities of the selected 
outcome equal to 1, Le., 


7, = lod ee 


The full model can be specified as follows: 
n, = B, + B, (Male, — Male.) + 8, (Hispanic, — Hispanic.)+ £, (Black, — Black.) 
+ B,(SpeEd, —SpeEd.)+ 2, (FRL, — FRL)+ £, (ELL, — ELL.) 
+ B,(GRD8ELA, — GRD8ELA.) + 8, (GRD8Math, - GRD8Math.) 
+ B,(OAGRD9, — OAGRD9.) + £,,(GRD9CRD, - GRD9CRD) 
+ B, (GRD9ADA, — GRD9ADA.)+ £,,(AVGGRD8ELA, — AVGGRD8ELA.) 
+ B,,(PFRL, —- PFRL)+ £,,(TRT) + ¢, 


where 


Y, represents the selected outcome for subject 7 


7, represents the logits of Pil ¥, = 1) 
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fy teptesents the mean logit for subject 7 adjusted for the covariates; 


BL, — £3 represent the logistic regression coefficients associated with various covariates 


for subject 7; 


f,, tepresents the logistic regression coefficient associated with the treatment indicator 


— it quantifies the treatment impact (the difference in the log-odds-ratio associated with 
being a treatment subject, as opposed to a comparison subject); 


€, represents the random error associated with subject 7. 


Table D.|. Summary of regression results for overall impacts on intended 
outcomes (201 I-13) 


Comparison Treatment Comparison Treatment 


Encolimentys 200 x 2 0.480 0.550 0.508 0.564 0.136 0.5396 
Discharge 
Graduation vs 
: 225 x2 0.564 0.582 0.532 0.626 0.234 0.2583 
Discharge 
ple UnadjustediMeane Regression-Adjusted e 
O O e Means ed e 
O O = ed Da r = 
Comparison Treatment Comparison Treatment De 
Average Daily 
Attendance SYI2 & 347 x2 60.055 65.372 60.043 65.383 5.340 0.248 | 0.0008 
SYI3 
ohn ee 348x2 0.167 0.080 0.129 0.128 | -0.002 | -0.003 0.9815 
Credits Earned 348 x2 16.733 19.926 16.252 20.556 4.304 0.369 | 0.0170 
Core Regents Passed 99x2 1.455 1.475 1.473 1.456 -0.016 | -0.012 0.9326 
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Table D.2. Summary of regression results for subgroup analyses of average daily 
attendance during school years 2012 and 2013 (201 I-13) 


bgroup p e 
Comparison Treatment Comparison Treatment i Delta 

Females 359 59.693 64.084 59.517 64.273 4.755 | 0.213 | 0.0367 

Males 335 60.473 66.652 60.613 66.523 5.910 | 0.285 0.0082 

Black Males 54 62.025 63.367 61.427 63.882 2.455 | 0.122 | 0.6623 

Hispanic Males 245 59.389 66.375 59.457 66.309 6.852 | 0.328 0.0078 

< II credits at admission 257 54.933 64.609 55.540 64.101 8.561 0.344 | 0.0027 


Table D.3. Summary of regression results for subgroup analyses of the number of 
suspensions during school years 2012 and 2013 (2011-13) 


bgroup p e 
Comparison Treatment Comparison Treatment _ Delta 

Females 357 0.130 0.069 0.153 0.068 -0.085 | -0.200 | 0.3287 

Males 339 0.207 0.091 0.165 0.131 -0.034 | -0.049 | 0.7009 

Black Males 56 0.185 0.172 0.184 0.173 -0.011 | -0.023 | 0.9282 

Hispanic Males 245 0.182 0.089 0.101 0.168 0.067 | 0.106 | 0.4778 

< II credits at admission 257 0.205 0.064 0.151 0.110 -0.041 | -0.060 | 0.6884 


Table D.4. Summary of regression results for subgroup analyses of the number of 
credits earned after admission (201 1-13) 


bgroup p e 
Comparison Treatment Comparison Treatment is Delta 

Females 357 16.973 20.872 16.155 21.744 5.589 | 0.479 | 0.0199 

Males 339 16.464 18.991 16.375 19.075 2.700 | 0.231 | 0.0370 

Black Males 56 15.788 18.638 16.964 17.357 0.393 | 0.032 | 0.9236 

Hispanic Males 245 15.820 18.772 15.839 18.754 2.915 | 0.259 | 0.0497 

< II credits at admission 257 15.513 21.622 16.160 21.474 5.314 | 0.410 | 0.0351 
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Table D.5. Summary of regression results for subgroup analyses of the number of 
core Regents exams passed after admission (201 I-13) 


bgroup p e 
Comparison Treatment Comparison Treatment 7 Delta 

Females 114 1.483 1.259 1.482 1.261 -0.222 | -0.153 | 0.4051 

Males 84 1.410 1.733 1.437 1.710 0.273 | 0.199 | 0.3527 

Black Males 10 

Hispanic Males 64 1.167 1.676 1.232 1.618 0.386 | 0.288 | 0.2753 

< II credits at admission 80 1.947 1.595 1.967 1.577 -0.390 | -0.233 | 0.2340 


Table D.6. Summary of regression results for subgroup analyses of enrollment vs. 
discharge status (201 1-13) 


Comparison Treatment Comparison Treatment 


Females 207 0.491 0.604 0.498 0.612 0.280 0.1100 
Males 193 0.468 0.495 0.521 0.499 -0.051 0.8610 
Black Males 27 

Hispanic Males 146 0.479 0.533 0.506 0.532 0.062 0.8053 
< II credits at admission 178 0.459 0.505 0.475 0.521 0.110 0.6970 


Table D.7. Summary of regression results for subgroup analyses of graduation vs. 
discharge status (201 I-13) 


Comparison Treatment Comparison Treatment 


Females 230 0.558 0.627 0.450 0.737 0.742 0.0292 
Males 220 0.571 0.539 0.574 0.538 -0.088 0.5991 
Black Males 4l 

Hispanic Males 15] 0.540 0.506 0.445 0.603 0.384 0.1910 
< I credits at admission 159 0.420 0.456 0.413 0.466 0.130 0.5222 
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Table D.8. Linear regression results of average daily attendance for all students, 
2011-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 

Male 1.180 1.642 0.720 0.4728 0.055 
Hispanic -3.308 2.415 -1.370 0.1711 -0.154 
Black -2.521 3.040 -0.830 0.4072 -0.117 
SpeEd 3.337 2.985 1.120 0.2640 0.155 
FRL -0.039 1.617 -0.020 0.9808 -0.002 
ELL -1.332 4.163 -0.320 0.7491 -0.062 
GRD8ELA -0.039 0.043 -0.910 0.3631 -0.002 
GRD8Math -0.024 0.039 -0.620 0.5367 -0.001 
OAGRD9 0.248 1.557 0.160 0.8736 0.011 

GRD9ICRD -0.257 0.227 -1.130 0.2568 -0.012 
GRD9ADA 0.330 0.062 5.320 <.0001 0.015 
AVGGRD8ELA 0.104 0.115 0.900 0.3659 0.005 
PFRL 0.014 0.149 0.090 0.9248 0.001 

TRT 3.269 2.540 1.290 0.1986 0.152 


Table D.9. Linear regression results of average daily attendance for all students, 
2011-13 (final model) 


Predictors | Estimate | SE t-ratio | 


Intercept 
GRD9ADA 0.303 0.051 5.970 <.0001 0.014 
TRT 5.340 1.584 3.370 0.0008 0.248 


Table D.10. HLM results of the number of suspensions for all students, 201 1-13 (full 
model) 


Fixed Effects Estimate | SE | less 
Delta 
Intercept 0.130 0.037 3.520 0.0024 -- 
Male 0.058 0.036 1.610 0.1077 0.101 
Hispanic -0.090 0.052 -1.710 0.0869 -0.158 
Black 0.015 0.067 0.230 0.8178 0.027 
SpeEd 0.061 0.064 0.950 0.3428 0.107 
FRL -0.029 0.035 -0.830 0.4094 -0.051 
ELL -0.032 0.090 -0.350 0.7231 -0.056 
GRD8ELA 0.000 0.001 -0.110 0.9110 0.000 
GRD8Math 0.001 0.001 0.700 0.4822 0.001 
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Fixed Effects | Estimate | SE | t-ratio Glassis 
Delta 
OAGRD9 0.015 0.6631 
GRDICRD -0.004 0.005 -0.740 0.4594 -0.006 
GRDIADA 0.000 0.001 -0.090 0.9248 0.000 
AVGGRD8ELA -0.005 0.003 -1.910 0.0653 -0.009 
PFRL 0.007 0.003 2.170 0.0346 0.013 
TRT -0.003 0.065 -0.050 0.9605 -0.006 
Random Effects Estimate | SE | z-value | p-value 
Between-School Variance 0.001 0.002 0.680 0.249 
Within-School Variance 0.206 0.011 18.550 <.0001 


Table D.I1. HLM results of the number of suspensions for all students, 201 1-13 
(final model) 


Glass’s 


Delta 


Fixed Effects | Estimate | SE | t-ratio 


Intercept 0.129 0.0024 
Male 0.063 0.035 1.810 0.0714 0.110 
Hispanic -0.100 0.038 -2.610 0.0092 -0.175 
AVGGRD8ELA -0.005 0.003 -1.920 0.0646 -0.009 
PFRL 0.007 0.003 2.070 0.0435 0.012 
TRT -0.002 0.065 -0.020 0.9815 -0.003 
Random Effects Estimate | SE | z-value | p-value 
Between-School Variance 0.001 0.002 0.740 0.231 
Within-School Variance 0.207 0.011 18.560 <.0001 


Table D.12. HLM results of the number of credits earned after admission for all 
students, 2011-13 (full model) 


Glass’s 


Delta 


Fixed Effects Estimate | SE | 


Intercept 

Male -1.350 0.930 -1.450 0.1470 -0.116 
Hispanic -0.963 1.363 -0.710 0.4804 -0.083 
Black -2.286 1.743 -1.310 0.1903 -0.196 
SpeEd -2.178 1.677 -1.300 0.1945 -0.187 
FRL 0.482 0.914 0.530 0.5977 0.041 
ELL -1.550 2.354 -0.660 0.5105 -0.133 
GRD8ELA -0.015 0.024 -0.600 0.5467 -0.001 
GRD8Math -0.017 0.022 -0.780 0.4352 -0.001 
OAGRD9 1.082 0.883 1.230 0.2207 0.093 
GRD9ICRD -0.394 0.147 -2.670 0.0077 -0.034 
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Fixed Effects 


Estimate | SE | 


t-ratio 


Glass’s 


Delta 


GRD9ADA 0.143 0.035 4.030 <.0001 0.012 
BeforeAdmission_CRD -0.113 0.056 -2.020 0.0437 -0.010 
AVGGRD8ELA -0.078 0.068 -1.150 0.2554 -0.007 
PFRL 0.168 0.085 1.960 0.0545 0.014 
TRT. 4.153 1.596 2.600 0.0222 0.356 


Random Effects Estimate SE z-value p-value 
Between-School Variance 0.511 0.964 0.530 0.298 
Within-School Variance 139.390 7.505 18.570 <.0001 


Table D.13. HLM results of the number of credits earned after admission for all 


students, 2011-13 (final model) 


Fixed Effects Estimate SE t-ratio p-value alas 
Delta 
Intercept 16.252 0.903 17.990 <.0001 -- 
Male -1.308 0.907 -1.440 0.1498 -0.112 
GRD9ICRD -0.398 0.144 -2.760 0.0059 -0.034 
GRD9ADA 0.142 0.035 4.090 <.0001 0.012 
BeforeAdmission_CRD -0.124 0.055 -2.250 0.0249 -0.01 1 
AVGGRD8ELA -0.091 0.067 -1.350 0.1834 -0.008 
PFRL 0.193 0.084 2.300 0.0249 0.017 
TRT 4.304 1.562 2.760 0.0170 0.369 
Random Effects Estimate SE z-value p-value 
Between-School Variance 0.410 0.921 0.450 0.328 
Within-School Variance 140.610 7.570 18.570 <.0001 


Table D.14. Linear regression results of the number of core Regents exams passed 


after admission for all students, 2011-13 (full model) 


Predictors 


Estimate | SE 


t-ratio 


Glass’s 


Delta 


Intercept 

Male 0.306 0.208 1.470 0.1435 0.217 
Hispanic -0.041 0.323 -0.130 0.8978 -0.029 
Black 0.376 0.382 0.990 0.3255 0.267 
SpeEd -0.941 0.321 -2.930 0.0038 -0.668 
FRL -0.365 0.210 -1.740 0.0844 -0.259 
ELL -0.822 0.549 -1.500 0.1364 -0.583 
GRD8ELA 0.012 0.006 1.980 0.0494 0.008 
GRD8Math 0.004 0.006 0.800 0.4221 0.003 
OAGRD9 0.006 0.185 0.030 0.9761 0.004 
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Predictors | Estimate | SE | t-ratio Glassis 
Delta 

GRDICRD 
GRD9ADA 0.002 0.009 0.250 0.8042 0.002 
BeforeAdmission_#Passed -0.479 0.113 -4.240 <.0001 -0.340 
AVGGRD8ELA -0.012 0.015 -0.790 0.4322 -0.009 
PFRL 0.007 0.016 0.450 0.6513 0.005 
TRT 0.188 0.316 0.590 0.5532 0.133 


Table D.15. Linear regression results of the number of core Regents passed after 
admission for all students, 201 1-13 (final model) 


Intercept 1.473 0.137 10.740 <.0001 -- 

Male 0.281 0.198 1.420 0.1573 0.199 
SpeEd -0.988 0.304 -3.250 0.0014 -0.701 
FRL -0.338 0.196 -1.720 0.0865 -0.240 
ELL -0.879 0.535 -1.640 0.1023 -0.624 
GRD8ELA 0.013 0.006 2.260 0.0250 0.009 
BeforeAdmission_#Passed -0.481 0.100 -4.820 <.0001 -0.341 
TRT -0.016 0.195 -0.080 0.9326 -0.012 


Table D.16. HGLM results of the enrollment vs. discharge status for all students, 
2011-13 (full model) 


Fixed Effects | Estimate | p-value Odds Ratio 
Intercept -0.051 0.241 -0.210 0.8380 -- 
Male -0.298 0.215 -1.390 0.1664 0.742 
Hispanic 0.047 0.326 0.140 0.8850 1.048 
Black 0.387 0.413 0.940 0.3497 1.473 
SpeEd 0.950 0.401 2.370 0.0183 2.584 
FRL 0.278 0.216 1.290 0.1991 1.321 
ELL -0.079 0.528 -0.150 0.8816 0.924 
GRD8ELA 0.005 0.006 0.840 0.4042 1.005 
GRD8Math 0.008 0.005 1.510 0.1331 1.008 
OAGRD9 -0.257 0.203 -1.270 0.2047 0.773 
GRDICRD -0.027 0.030 -0.890 0.3717 0.973 
GRD9ADA 0.015 0.008 1.850 0.0644 1.015 
AVGGRD8ELA -0.001 0.018 -0.060 0.9488 0.999 
PFRL -0.028 0.022 -1.260 0.2083 0.972 
TRT 0.321 0.429 0.750 0.4552 1.378 

Random Component Estimate | SE 
Intercept 0.066 0.095 
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Table D.17. HGLM results of the enrollment vs. discharge status for all students, 
2011-13 (final model) 


Fixed Effects | Estimate | t-ratio p-value Odds Ratio 
Intercept 0.031 0.199 0.160 0.8795 -- 
Male -0.312 0.207 -1.510 0.1328 0.732 
SpeEd 0.757 0.377 2.010 0.0450 2.132 
GRD8Math 0.008 0.005 1.700 0.0901 1.008 
GRD9ADA 0.009 0.006 1.470 0.1418 1.009 
TRT 0.224 0.366 0.610 0.5396 1.252 

Random Component Estimate | SE 
Intercept 0.144 0.148 


Table D.18. HGLM results of the graduation vs. discharge status for all students, 
2011-13 (full model) 


Fixed Effects | Estimate t-ratio Odds Ratio 
Intercept 0.077 0.222 0.350 0.7369 -- 
Male -0.203 0.207 -0.980 0.3272 0.817 
Hispanic -0.070 0.292 -0.240 0.8114 0.933 
Black -0.228 0.389 -0.590 0.5578 0.796 
SpeEd 0.298 0.433 0.690 0.4908 1.348 
FRL 0.194 0.201 0.970 0.3345 1.214 
ELL -0.530 0.499 -1.060 0.289 0.589 
GRD8ELA -0.005 0.005 -0.950 0.3409 0.995 
GRD8Math 0.005 0.005 1.080 0.2810 1.005 
OAGRD9 0.092 0.200 0.460 0.6456 1.097 
GRD9ICRD 0.024 0.029 0.820 0.4103 1.024 
GRD9ADA 0.020 0.008 2.480 0.0135 1.020 
AVGGRD8ELA -0.023 0.017 -1.390 0.1662 0.977 
PFRL 0.007 0.020 0.360 0.7223 1.007 
TRT 0.499 0.398 1.260 0.2101 1.647 

Random Component Estimate | SE 
Intercept 0.060 0.163 


Table D.19. HGLM results of the graduation vs. discharge status for all students, 
2011-13 (final model) 


Intercept 0.129 0.197 0.650 0.5282 -- 

GRD9ADA 0.023 0.006 3.490 0.0005 1.023 

AVGGRD8ELA -0.018 0.012 -1.450 0.1485 0.982 

TRT 0.386 0.341 1.130 0.2583 1.472 
Random Component Estimate | SE 

Intercept 0.047 0.113 
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Table D.20. Linear regression results of average daily attendance for female 
students, 2011-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 

Hispanic -2.899 3.237 -0.900 0.3710 -0.130 
Black -1.745 4.164 -0.420 0.6754 -0.078 
SpeEd 1.326 4.716 0.280 0.7787 0.060 
FRL -0.325 2.384 -0.140 0.8915 -0.015 
ELL 3.945 7.238 0.540 0.5861 0.177 
GRD8ELA -0.066 0.064 -1.030 0.3050 -0.003 
GRD8Math -0.039 0.062 -0.640 0.5216 -0.002 
OAGRD9 2.126 2.481 0.860 0.3921 0.095 
GRD9ICRD -0.174 0.324 -0.540 0.5918 -0.008 
GRD9ADA 0.344 0.090 3.830 0.0002 0.015 
AVGGRD8ELA 0.139 0.182 0.770 0.4442 0.006 
PFRL -0.069 0.232 -0.300 0.7668 -0.003 
TRT 1.830 3.953 0.460 0.6438 0.082 


Table D.21. Linear regression results of average daily attendance for female 
students, 2011-13 (final model) 


Glass’s 


Delta 


Predictors Estimate | SE | p-value | 


Intercept 
GRD9ADA 0.316 0.073 4.350 <.0001 0.014 
TRT 4.755 2.267 2.100 0.0367 0.213 


Table D.22. Linear regression results of average daily attendance for male students, 
2011-13 (full model) 


Predictors | Estimate | SE | 


Intercept 

Hispanic -4.715 3.777 -1.250 0.2128 -0.228 
Black -3.962 4.616 -0.860 0.3914 -0.191 
SpeEd 4.976 3.998 1.240 0.2142 0.240 
FRL -0.311 2.299 -0.140 0.8923 -0.015 
ELL -3.736 5.121 -0.730 0.4662 -0.180 
GRD8ELA -0.001 0.060 -0.020 0.9815 0.000 
GRD8Math -0.014 0.051 -0.270 0.7857 -0.001 
OAGRD9 -1.091 2.050 -0.530 0.5951 -0.053 
GRD9ICRD -0.386 0.325 -1.190 0.2363 -0.019 
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Predictors | Estimate | SE | t-ratio | Glassis 
Delta 
GRDIADA 
AVGGRD8ELA 0.070 0.153 0.460 0.6460 0.003 
PFRL 0.077 0.202 0.380 0.7026 0.004 
TRT 4.312 3.430 1.260 0.2095 0.208 


Table D.23. Linear regression results of average daily attendance for male students, 
2011-13 (final model) 


Predictors | Estimate | SE | Classis 
Delta 

Intercept 60.613 1.600 37.880 <.0001 -- 
GRDIADA 0.283 0.071 3.980 <.0001 0.014 
TRT 5.910 2.221 2.660 0.0082 0.285 


Table D.24. Linear regression results of average daily attendance for Black male 
students, 2011-13 (full model) 


Delta 


Predictors | Estimate | SE | t-ratio p-value 


| Glass’s 


Intercept 

SpeEd -3.504 11.558 -0.300 0.7632 -0.175 
FRL -9.235 6.211 -1.490 0.1444 -0.461 
ELL 0.000 -- -- -- -- 
GRD8ELA -0.101 0.195 -0.520 0.6062 -0.005 
GRD8Math 0.019 0.149 0.120 0.9016 0.001 
OAGRD9Y -4.955 6.542 -0.760 0.4529 -0.247 
GRD9ICRD 0.202 1.119 0.180 0.8578 0.010 
GRD9ADA 0.324 0.253 1.280 0.2074 0.016 
AVGGRD8ELA 0.030 0.397 0.070 0.9407 0.001 
PFRL 0.185 0.442 0.420 0.6779 0.009 
TRT -1.460 9.986 -0.150 0.8844 -0.073 


Table D.25. Linear regression results of average daily attendance for Black male 
students, 2011-13 (final model) 


Glass’s 


Predictors Estimate 


Delta 


Intercept 
GRD9ADA -11.056 5.637 -1.960 0.0553 -0.552 
TRT 2.455 5.590 0.440 0.6623 0.122 
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Table D.26. Linear regression results of average daily attendance for Hispanic male 
students, 2011-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 60.291 2.356 25.600 <.0001 

SpeEd 7.432 4.45] 1.670 0.0963 0.356 
FRL 0.749 2.651 0.280 0.7778 0.036 
ELL -3.256 5.088 -0.640 0.5229 -0.156 
GRD8ELA 0.056 0.070 0.810 0.4216 0.003 
GRD8Math -0.047 0.060 -0.790 0.4318 -0.002 
OAGRD9 0.312 2.342 0.130 0.8942 0.015 
GRD9ICRD -0.433 0.368 -1.180 0.2403 -0.021 
GRD9ADA 0.306 0.097 3.160 0.0018 0.015 
AVGGRD8ELA 0.076 0.174 0.440 0.6619 0.004 
PFRL 0.110 0.255 0.430 0.6671 0.005 
TRT. 5.204 3.899 1.330 0.1833 0.249 


Table D.27. Linear regression results of average daily attendance for Hispanic male 
students, 2011-13 (final model) 


Glass’s 


Predictors Estimate t-ratio p-value 
Delta 


Intercept 59.457 32.760 <.0001 - 
[SepE¢d i t:—~—“—*é‘sYSC‘éwd | 4074 T7000 0.0908 0332] 

GRD9ADA 0.248 0.080 3.110 0.0021 0.012 

TRT 6.852 2.552 2.690 0.0078 0.328 


Table D.28. Linear regression results of average daily attendance for students with 
less than I 1 credits at admission, 201 1-13 (full model) 


Predictors Estimate SE t-ratio p-value Ges: 
Delta 
Intercept 56.164 3.009 18.660 <.0001 -- 

[Male | 2351 | 2990 | 0790 | 04324 | 0094 | 
[Hispanic | -4887 | 4385 | -L.I1O | 0.2662 | -0196 | 
[Black | -4392 [| 5736 | -0770 | 04446 | -0.176 | 

SpeEd 6.029 5.933 1.020 0.3106 0.242 

FRL -0.936 2.988 -0.310 0.7544 -0.038 

ELL -5.177 7.640 -0.680 0.4987 -0.208 

GRD8ELA -0.047 0.086 -0.550 0.5832 -0.002 

GRD8Math -0.044 0.075 -0.580 0.5626 -0.002 

OAGRD9 -2.948 2.868 -1.030 0.3051 -0.118 

GRD9ICRD -1.049 0.629 -1.670 0.0964 -0.042 
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Predictors | Estimate | SE | t-ratio Glassis 
Delta 
GRDIADA 
AVGGRD8ELA 0.084 0.205 0.410 0.6818 0.003 
PFRL -0.057 0.257 -0.220 0.8242 -0.002 
TRT 7416 4.874 1.520 0.1294 0.298 


Table D.29. Linear regression results of average daily attendance for students with 


less than I 1 credits at admission, 201 1-13 (final model) 


Predictors | Estimate | SE p-value | polasss 
Delta 

Intercept 55.540 2.081 26.680 <.0001 -- 
GRDICRD -1.327 0.599 -2.220 0.0276 -0.053 
GRD9ADA 0.426 0.089 4.800 <.0001 0.017 
TRT 8.561 2.824 3.030 0.0027 0.344 


Table D.30. HLM results of the number of suspensions for female students, 201 1-13 
(full model) 


Fixed Effects | Estimate | SE | telessis 
Delta 
Intercept 0.112 0.046 2.450 0.0370 -- 
Hispanic -0.133 0.051 -2.590 0.0099 -0.315 
Black 0.040 0.068 0.580 0.5617 0.093 
SpeEd 0.061 0.074 0.820 0.4139 0.143 
FRL 0.009 0.038 0.240 0.8102 0.021 
ELL 0.116 0.115 1.010 0.3112 0.274 
GRD8ELA -0.001 0.001 -1.240 0.2155 -0.003 
GRD8Math 0.002 0.001 1.830 0.0679 0.004 
OAGRD9 0.026 0.039 0.650 0.5147 0.061 
GRD9ICRD -0.012 0.005 -2.390 0.0174 -0.029 
GRD9ADA 0.000 0.001 0.090 0.9246 0.000 
AVGGRD8ELA -0.004 0.003 -1.140 0.2769 -0.009 
PFRL 0.007 0.004 1.670 0.1110 0.017 
TRT -0.008 0.087 -0.100 0.9276 -0.020 
Rando e a alue D-va 
Between-School Variance 0.004 0.005 0.790 0.215 
Within-School Variance 0.115 0.009 13.000 <.0001 
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Table D.31. HLM results of the number of suspensions for female students, 201 1-13 


(final model) 


Fixed Effects 


Estimate | SE | 


Glass’s 


Delta 


Random Effects 


Between-School Variance 


Estimate 
0.007 


z-value 
1.100 


Intercept 0.153 0.0029 

Hispanic -0.153 0.040 -3.820 0.0002 -0.361 
GRDICRD -0.011 0.004 -2.550 0.0113 -0.026 
TRT -0.085 0.077 -1.100 0.3287 -0.200 


Within-School Variance 


0.116 


13.020 


Table D.32. Linear regression results of the number of suspensions for male 
students, 2011-13 (full model) 


Predictors 


Estimate | SE | 


Glass’s 


Delta 


Intercept 

Hispanic -0.029 0.099 -0.290 0.7698 -0.042 
Black 0.021 0.122 0.170 0.8620 0.030 
SpeEd 0.092 0.107 0.860 0.3904 0.132 
FRL -0.091 0.062 -1.480 0.1412 -0.131 
ELL -0.146 0.138 -1.050 0.2928 -0.210 
GRD8ELA 0.001 0.002 0.910 0.3639 0.002 
GRD8Math -0.001 0.001 -0.420 0.6782 -0.001 
OAGRD9 0.008 0.055 0.150 0.8826 0.012 
GRD9ICRD 0.007 0.009 0.770 0.4392 0.010 
GRD9ADA 0.000 0.002 -0.140 0.8875 0.000 
AVGGRD8ELA -0.006 0.004 - 1.400 0.1612 -0.008 
PFRL 0.007 0.005 1.390 0.1647 0.010 
TRT -0.040 0.091 -0.440 0.6584 -0.058 


Table D.33. Linear regression results of the number of suspensions for male 
students, 2011-13 (final model) 


Predictors 


Estimate | SE | 


t-ratio 


Glass’s 


Delta 


Intercept 

AVGGRD8ELA -0.005 0.004 -1.370 0.1713 -0.008 
PFRL 0.007 0.005 1.380 0.1692 0.010 
TRT. -0.034 0.089 -0.380 0.7009 -0.049 
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Table D.34. Linear regression results of the number of suspensions for Black male 
students, 2011-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 

SpeEd -0.048 0.237 -0.200 0.8422 -0.098 
FRL -0.006 0.135 -0.040 0.9663 -0.012 
ELL 0.000 -- -- -- -- 
GRD8ELA -0.003 0.004 -0.840 0.4079 -0.007 
GRD8Math 0.001 0.003 0.170 0.8637 0.001 
OAGRD9 -0.060 0.145 -0.420 0.6786 -0.125 
GRD9ICRD 0.039 0.024 1.590 0.1179 0.080 
GRD9ADA -0.012 0.005 -2.240 0.0299 -0.025 
AVGGRD8ELA 0.006 0.008 0.710 0.4821 0.012 
PFRL 0.004 0.009 0.390 0.7009 0.007 
TRT -0.181 0.208 -0.870 0.3904 -0.374 


Table D.35. Linear regression results of the number of suspensions for Black male 
students, 2011-13 (final model) 


Glass’s 


Delta 


Predictors Estimate | SE | t-ratio 


Intercept 

GRD9ICRD 0.040 0.019 2.130 0.0380 0.082 
GRD9ADA -0.012 0.005 -2.650 0.0106 -0.025 
TRT -0.011 0.121 -0.090 0.9282 -0.023 


Table D.36. Linear regression results of the number of suspensions for Hispanic 
male students, 201 1-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 

SpeEd 0.149 0.112 1.330 0.1863 0.236 
FRL -0.113 0.067 -1.690 0.0924 -0.179 
ELL -0.117 0.129 -0.910 0.3650 -0.184 
GRD8ELA 0.003 0.002 1.870 0.0631 0.005 
GRD8Math -0.001 0.002 -0.580 0.5609 -0.001 
OAGRD9 -0.001 0.059 -0.020 0.9809 -0.002 
GRD9ICRD 0.011 0.009 1.190 0.2334 0.018 
GRD9ADA 0.001 0.002 0.230 0.8221 0.001 

AVGGRD8ELA -0.010 0.004 -2.160 0.0316 -0.015 
PFRL 0.001 0.006 0.190 0.8504 0.002 
TRT 0.071 0.098 0.720 0.4723 0.112 
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male students, 201 1-13 (final model) 


Predictors 


| Estimate | SE | 


Glass’s 


Delta 


Intercept 

SpeEd 0.147 0.110 1.340 0.1827 0.232 
FRL -0.106 0.066 -1.600 O.1TTI -0.168 
GRD8ELA 0.003 0.001 2.090 0.0377 0.005 
AVGGRD8ELA -0.009 0.004 -2.440 0.0155 -0.015 
TRT 0.067 0.095 0.710 0.4778 0.106 


Table D.38. Linear regression results of the number of suspensions for students with 
less than I 1 credits at admission, 201 1-13 (full model) 


Predictors | Estimate | SE | t-ratio | Sisss 
Delta 
Intercept 0.129 0.065 1.980 0.0486 -- 
Male 0.041 0.066 0.620 0.5330 0.061 
Hispanic -0.105 0.096 - 1.090 0.2786 -0.155 
Black -0.072 0.126 -0.570 0.5685 -0.107 
SpeEd 0.095 0.128 0.740 0.4588 0.140 
FRL -0.043 0.066 -0.650 0.5168 -0.063 
ELL -0.043 0.168 -0.260 0.7975 -0.064 
GRD8ELA -0.001 0.002 -0.750 0.4549 -0.002 
GRD8Math 0.001 0.002 0.840 0.4007 0.002 
OAGRD9 0.027 0.063 0.430 0.6710 0.040 
GRDICRD -0.017 0.014 - 1.260 0.2106 -0.026 
GRDIADA 0.001 0.002 0.610 0.5392 0.002 
AVGGRD8ELA -0.008 0.004 - 1.900 0.0583 -0.012 
PFRL 0.015 0.006 2.640 0.0088 0.021 
TRT -0.001 0.105 -0.010 0.9888 -0.002 


Table D.39. Linear regression results of the number of suspensions for students with 
less than I 1 credits at admission, 201 1-13 (final model) 


Predictors Estimate | SE iclessis 
Delta 

Intercept 0.151 0.063 2.380 0.0179 -- 
AVGGRD8ELA -0.007 0.004 -1.720 0.0868 -0.011 
PFRL 0.014 0.005 2.630 0.0091 0.020 
TRT -0.041 0.101 -0.400 0.6884 -0.060 
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Table D.40. HLM results of the number of credits earned after admission for female 
students, 2011-13 (full model) 


Fixed Effects 


Estimate | SE | 


Glass’s 


Delta 


Intercept 16.077 13.010 <.0001 
Hispanic 0.105 1.783 0.060 0.9529 0.009 
Black -2.188 2.337 -0.940 0.3499 -0.188 
SpeEd -3.341 2.596 -1.290 0.1990 -0.286 
FRL -0.776 1.316 -0.590 0.5559 -0.066 
ELL -0.98 | 3.978 -0.250 0.8053 -0.084 
GRD8ELA -0.020 0.035 -0.550 0.5805 -0.002 
GRD8Math -0.041 0.034 -1.210 0.2278 -0.004 
OAGRD9 1.072 1.379 0.780 0.4374 0.092 
GRD9ICRD -0.210 0.208 -1.010 0.3128 -0.018 
GRD9ADA 0.121 0.050 2.440 0.0153 0.010 
BeforeAdmission_CRD -0.185 0.075 -2.460 0.0144 -0.016 
AVGGRD8ELA -0.159 0.099 -1.610 0.1148 -0.014 
PFRL 0.263 0.125 2.110 0.0398 0.023 
TRT 5.774 2.193 2.630 0.0211 0.495 
Random Effects Estimate SE z-value p-value 
Between-School Variance 0.198 1.678 0.120 0.453 
Within-School Variance 140.420 10.626 13.220 <.0001 


Table D.41. HLM results of the number of credits earned after admission for female 
students, 2011-13 (final model) 


Fixed Effects Estimate | SE | p-value | talassis 
Delta 
Intercept 16.155 1.218 13.270 <.0001 -- 
GRD9ADA 0.102 0.044 2.300 0.0218 0.009 
BeforeAdmission_CRD -0.242 0.064 -3.770 0.0002 -0.021 
AVGGRD8ELA -0.133 0.097 -1.370 0.1757 -0.01 1 
PFRL 0.247 0.122 2.020 0.0478 0.021 
TRT 5.589 2.146 2.600 0.0199 0.479 
Random Effects Estimate SE z-value p-value 
Between-School Variance 0.015 1.432 0.010 0.496 
Within-School Variance 143.320 10.822 13.240 <.0001 
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Table D.42. Linear regression results of the number of credits earned after 
admission for male students, 2011-13 (full model) 


Predictors 


Estimate | SE 


Glass’s 


Delta 


Intercept 

Hispanic -2.955 2.173 -1.360 0.1749 -0.253 
Black -3.225 2.676 -1.200 0.2291 -0.276 
SpeEd -1.366 2.307 -0.590 0.5540 -0.117 
FRL 1.243 1.341 0.930 0.3546 0.107 
ELL -1.737 2.992 -0.580 0.5620 -0.149 
GRD8ELA -0.007 0.035 -0.210 0.8318 -0.001 
GRD8Math 0.001 0.030 0.020 0.9819 0.000 
OAGRD9Y 1.025 1.194 0.860 0.3913 0.088 
GRD9ICRD -0.596 0.215 -2.770 0.0058 -0.051 
GRD9ADA 0.152 0.052 2.950 0.0035 0.013 
BeforeAdmission_CRD -0.008 0.085 -0.090 0.9251 -0.001 
AVGGRD8ELA -0.058 0.088 -0.660 0.5084 -0.005 
PFRL 0.135 0.113 1.190 0.2331 0.012 
TRT. 3.442 1.969 1.750 0.0814 0.295 


Table D.43. Linear regression results of the number of credits earned after 
admission for male students, 201 1-13 (final model) 


Estimate | SE | 


Predictors t-ratio 
Intercept 16.375 0.926 17.690 <.0001 -- 
GRDICRD -0.592 0.182 -3.260 0.0012 -0.051 
GRDIADA 0.155 0.049 3.130 0.0019 0.013 
TRT 2.700 1.289 2.090 0.0370 0.231 


Table D.44. HLM results of the number of credits earned after admission for Black 


male students, 201 1-13 (full model) 


Fixed Effects 


| Estimate | SE | 


t-ratio 


Glass’s 


Delta 


Intercept 

SpeEd -7.568 5.244 -1.440 0.1545 -0.616 
FRL -0.149 2.976 -0.050 0.9602 -0.012 
ELL 0.000 -- ~ = = 
GRD8ELA -0.100 0.096 - 1.040 0.3047 -0.008 
GRD8Math 0.069 0.069 1.000 0.3223 0.006 
OAGRD9 1.056 3.163 0.330 0.7400 0.086 
GRD9ICRD -0.900 0.578 -1.560 0.1258 -0.073 
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Fixed Effects 


Estimate | SE 


t-ratio 


Glass’s 


Delta 


GRD9ADA 0.460 0.120 3.840 0.0003 0.037 
BeforeAdmission_CRD 0.092 0.169 0.540 0.5880 0.008 
AVGGRD8ELA -0.054 0.199 -0.270 0.7884 -0.004 
PFRL 0.401 0.222 1.800 0.0849 0.033 
TRT. 0.886 5.181 0.170 0.8673 0.072 


Random Effects Estimate SE z-value p-value 
Between-School Variance 6.477 12.241 0.530 0.298 
Within-School Variance 105.200 21.576 4.880 <.0001 


Table D.45. HLM results of the number of credits earned after admission for Black 


male students, 201 1-13 (final model) 


Fixed Effects Estimate SE t-ratio p-value alas 
Delta 
Intercept 16.964 2.405 7.050 <.0001 -- 
SpeEd -8.148 5.161 -1.580 0.1201 -0.663 
GRD9ICRD -0.665 0.459 -1.450 0.1531 -0.054 
GRD9ADA 0.439 0.112 3.930 0.0002 0.036 
PFRL 0.263 0.188 1.400 0.1781 0.021 
TRT 0.393 3.968 0.100 0.9236 0.032 
Random Effects Estimate SE z-value p-value 
Between-School Variance 10.373 12.197 0.850 0.198 
Within-School Variance 107.280 21.510 4.990 <.0001 


Table D.46. Linear regression results of the number of credits earned after 
admission for Hispanic male students, 201 1-13 (full model) 


Predictors 


Estimate 


t-ratio 


Glass’s 


Delta 


Intercept 

SpeEd -0.196 2.617 -0.070 0.9403 -0.017 
FRL 0.684 1.558 0.440 0.6609 0.061 

ELL -1.929 2.992 -0.640 0.5197 -0.171 
GRD8ELA 0.002 0.041 0.050 0.9581 0.000 
GRD8Math -0.002 0.035 -0.060 0.9499 0.000 
OAGRD9 1.593 1.375 1.160 0.2480 0.141 

GRD9ICRD -0.551 0.241 -2.290 0.0229 -0.049 
GRD9ADA 0.120 0.058 2.060 0.0406 0.011 

BeforeAdmission_CRD -0.069 0.100 -0.690 0.4925 -0.006 
AVGGRD8ELA -0.019 0.103 -0.180 0.8575 -0.002 
PFRL 0.006 0.150 0.040 0.9671 0.001 

TRT 3.051 2.322 1.310 0.1902 0.271 
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Table D.47. Linear regression results of the number of credits earned after 
admission for Hispanic male students, 201 1-13 (final model) 


Predictors 


Estimate 


p-value | 


Glass’s 


Delta 


Intercept 

GRD9ICRD -0.610 0.207 -2.950 0.0035 -0.054 
GRD9ADA 0.109 0.056 1.950 0.0521 0.010 
TRT 2.915 1.478 1.970 0.0497 0.259 


Table D.48. HLM results of the number of credits earned after admission for 
students with less than || credits at admission, 2011-13 (full model) 


Fixed Effects | Estimate t-ratio 

Intercept 16.206 1.809 8.960 <.0001 -- 
Male - 1.844 1.742 - 1.060 0.2909 -0.142 
Hispanic -0.721 2.566 -0.280 0.7790 -0.056 
Black -7.348 3.402 -2.160 0.0317 -0.567 
SpeEd -0.887 3.383 -0.260 0.7934 -0.068 
FRL 0.303 1.733 0.170 0.8612 0.023 
ELL -5.643 4.430 -1.270 0.2039 -0.436 
GRD8ELA -0.021 0.050 -0.420 0.6781 -0.002 
GRD8Math -0.072 0.044 - 1.640 0.1017 -0.006 
OAGRD9 1.154 1.702 0.680 0.4982 0.089 
GRD9ICRD -1.285 0.424 -3.030 0.0027 -0.099 
GRD9ADA 0.138 0.056 2.460 0.0146 0.011 
BeforeAdmission_CRD 1.034 0.344 3.000 0.0029 0.080 
AVGGRD8ELA -0.012 0.121 -0.100 0.9230 -0.001 
PFRL 0.104 0.150 0.690 0.4922 0.008 
TRT 5.102 2.997 1.700 0.1069 0.394 

Random Effects Estimate | z-value 
Between-School Variance 1.165 0.410 
Within-School Variance 173.030 15.423 11.220 <.0001 


Table D.49. HLM results of the number of credits earned after admission for 
students with less than I 1 credits at admission, 2011-13 (final model) 


Fixed Effects | Estimate less 
Delta 

Intercept 16.160 1.347 12.000 <.0001 -- 
Black -6.802 2.335 -2.910 0.0039 -0.525 
GRD8Math -0.08 | 0.039 -2.050 0.0418 -0.006 
GRD9ICRD -1.257 0.411 -3.060 0.0024 -0.097 
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t-ratio 


0.0159 


Glass’s 
Delta 


BeforeAdmission_CRD 


1.010 


0.0029 


TRT. 


Random Effects 


Between-School Variance 


5.314 


Estimate 
2.037 


0.0351 
z-value 
0.600 


Within-School Variance 


175.170 


11.210 


Table D.50. Linear regression results of the number of core Regents exams passed 
after admission for female students, 2011-13 (full model) 


Predictors Estimate t-ratio Giese 
Delta 
Intercept 1.350 0.251 5.380 <.0001 -- 
Hispanic 0.209 0.457 0.460 0.6484 0.145 
Black 0.641 0.521 1.230 0.2212 0.444 
SpeEd -0.796 0.501 -1.590 0.1151 -0.551 
FRL -0.101 0.301 -0.340 0.7365 -0.070 
ELL -0.917 0.897 - 1.020 0.3091 -0.635 
GRD8ELA 0.009 0.008 1.110 0.2689 0.006 
GRD8Math 0.006 0.008 0.650 0.5184 0.004 
OAGRD9 -0.114 0.277 -0.410 0.6828 -0.079 
GRD9ICRD 0.007 0.039 0.180 0.8561 0.005 
GRD9ADA -0.004 0.013 -0.340 0.7331 -0.003 
BeforeAdmission_#Passed -0.547 0.159 -3.440 0.0008 -0.379 
AVGGRD8ELA -0.015 0.021 -0.700 0.4829 -0.010 
PFRL -0.003 0.021 -0.160 0.8750 -0.002 
TRT 0.056 0.448 0.130 0.9002 0.039 


Table D.51. Linear regression results of the number of core Regents passed after 
admission for female students, 201 1-13 (final model) 


Predictors 


Estimate | SE | 


t-ratio 


Intercept 

SpeEd - 1.033 0.431 -2.390 0.0183 -0.716 
BeforeAdmission_#Passed -0.430 0.130 -3.300 0.0013 -0.298 
TRT -0.222 0.265 -0.840 0.4051 -0.153 
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Table D.52. Linear regression results of the number of core Regents exams passed 
after admission for male students, 2011-13 (full model) 


Glass’s 


Predictors Estimate SE 
Delta 


Intercept 

Hispanic -0.167 0.484 -0.340 0.7317 -0.121 
Black 0.216 0.669 0.320 0.7482 0.157 
SpeEd -0.955 0.445 -2.150 0.0354 -0.696 
FRL -0.702 0.316 -2.220 0.0296 -0.512 
ELL -0.863 0.719 -1.200 0.2341 -0.630 
GRD8ELA 0.013 0.009 1.540 0.1279 0.010 
GRD8Math 0.001 0.008 0.170 0.8671 0.001 

OAGRD9Y 0.064 0.274 0.230 0.8171 0.046 
GRD9ICRD -0.087 0.045 -1.940 0.0571 -0.064 
GRD9ADA 0.011 0.015 0.730 0.4694 0.008 
BeforeAdmission_#Passed -0.450 0.173 -2.600 0.0113 -0.328 
AVGGRD8ELA -0.009 0.026 -0.340 0.7317 -0.006 
PFRL 0.030 0.026 1.160 0.2504 0.022 
TRT 0.366 0.484 0.760 0.4522 0.267 


Table D.53. Linear regression results of the number of core Regents passed after 
admission for male students, 201 1-13 (final model) 


Predictors Estimate | SE | t-ratio 

Intercept 1.437 0.211 6.800 <.0001 -- 
SpeEd - 1.028 0.415 -2.480 0.0155 -0.749 
FRL -0.708 0.292 -2.430 0.0175 -0.516 
GRD8ELA 0.016 0.008 1.980 0.0513 0.011 
GRDICRD -0.065 0.036 - 1.820 0.0734 -0.048 
BeforeAdmission_#Passed -0.422 0.159 -2.650 0.0098 -0.308 
TRT 0.273 0.292 0.940 0.3527 0.199 


Table D.54. Linear regression results of the number of core Regents exams passed 
after admission for Hispanic male students, 201 1-13 (full model) 


Glass’s 


Delta 


Predictors Estimate t-ratio | 


Intercept 

SpeEd -0.615 0.525 -1.170 0.2470 -0.458 
FRL -0.805 0.384 -2.100 0.0407 -0.601 
ELL -0.871 0.759 -1.150 0.2563 -0.650 
GRD8ELA 0.015 0.011 1.410 0.1636 0.011 
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Predictors | Estimate | SE | t-ratio Glassis 
Delta 
GRD8Math 
OAGRD9 -0.064 0.327 -0.190 0.8465 -0.047 
GRDICRD -0.082 0.054 -1.520 0.1341 -0.061 
GRD9ADA 0.011 0.016 0.640 0.5224 0.008 
BeforeAdmission_#Passed -0.480 0.207 -2.320 0.0243 -0.358 
AVGGRD8ELA -0.014 0.031 -0.450 0.6518 -0.010 
PFRL 0.035 0.032 1.090 0.2803 0.026 
TRT 0.511 0.549 0.930 0.3563 0.381 


Table D.55. Linear regression results of the number of core Regents passed after 
admission for Hispanic male students, 201 1-13 (final model) 


Predictors Estimate t-ratio 
Intercept 1.232 0.253 4.870 <.0001 -- 
FRL -0.736 0.352 -2.090 0.0407 -0.548 
GRD8ELA 0.018 0.010 1.910 0.0608 0.014 
BeforeAdmission_#Passed -0.540 0.179 -3.010 0.0038 -0.403 
TRT 0.386 0.351 1.100 0.2753 0.288 


Table D.56. Linear regression results of the number of core Regents exams passed 
after admission for students with less than I | credits at admission, 2011-13 (full 
model) 


Predictors Estimate | SE | Sissi 
Delta 
Intercept 1.871 0.325 5.760 <.0001 -- 
[Male | 0454 | 0347 | 1310 | £01954 #| O271 | 
[Hispanic | -O110 | O485 | -0230 | 08213 | 0066 | 
[Black | 0300 | O61 | 0490 | 406259 | 0179 | 
SpeEd - 1.402 0.551 -2.540 0.0134 -0.837 
FRL -0.689 0.349 -1.970 0.0527 -0.411 
ELL -1.074 0.846 -1.270 0.2089 -0.641 
GRD8ELA 0.015 0.012 1.300 0.1984 0.009 
GRD8Math 0.003 0.011 0.280 0.7811 0.002 
OAGRD9 -0.140 0.333 -0.420 0.6747 -0.084 
GRD9ICRD -0.100 0.067 - 1.490 0.1414 -0.060 
GRD9ADA 0.017 0.014 1.240 0.2212 0.010 
BeforeAdmission_#Passed -0.918 0.344 -2.670 0.0096 -0.548 
AVGGRD8ELA -0.008 0.026 -0.310 0.7563 -0.005 
PFRL -0.003 0.025 -0.120 0.9068 -0.002 
TRT -0.207 0.539 -0.380 0.7024 -0.123 
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Table D.57. Linear regression results of the number of core Regents passed after 
admission for students with less than I | credits at admission, 2011-13 (final model) 


Predictors Estimate t-ratio 


Delta 


| Glass’s 


Intercept 

[Speed |  -1077 | O495 | -2180 | 00326 | -0643 | 
FRL -0.637 0.325 - 1.960 0.0535 -0.380 
GRD8ELA 0.023 0.010 2.370 0.0204 0.014 
BeforeAdmission_#Passed -0.897 0.323 -2.780 0.0069 -0.535 
TRT -0.390 0.325 -1.200 0.2340 -0.233 


Table D.58. Logistic regression results of the enrollment vs. discharge status for 
female students, 2011-13 (full model) 


Predictors | Estimate Weld: p-value 
Square 

Intercept 0.006 0.293 0.000 0.9846 -- 

Hispanic 0.079 0.415 0.036 0.849 | 1.082 
Black 1.47] 0.559 6.926 0.0085 4.353 
SpeEd 0.623 0.618 1.015 0.3136 1.864 
FRL 0.257 0.320 0.643 0.4225 1.293 
ELL 0.337 0.923 0.133 0.7152 1.400 
GRD8ELA 0.010 0.009 1.231 0.2673 1.010 
GRD8Math -0.002 0.008 0.088 0.7669 0.998 
OAGRD9 -0.034 0.315 0.012 0.9130 0.966 
GRD9ICRD 0.002 0.042 0.003 0.9541 1.002 
GRD9ADA 0.013 0.011 1.341 0.2469 1.013 
AVGGRD8ELA 0.001 0.025 0.001 0.9750 1.001 
PFRL -0.040 0.032 1.543 0.2142 0.961 
TRT 0.456 0.512 0.792 0.3734 1.577 


Table D.59. Logistic regression results of the enrollment vs. discharge status for 
female students, 2011-13 (final model) 


Predictors Estimate | Welds p-value Odds Ratio 
Square 
Intercept -0.007 0.202 0.001 0.9724 -- 
Black 1.404 0.428 10.782 0.0010 4.073 
TRT 0.463 0.290 2.555 0.1100 1.589 
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Table D.60. HGLM results of the enrollment vs. discharge status for male students, 
2011-13 (full model) 


Fixed Effects | Estimate | t-ratio p-value Odds Ratio 
Intercept 0.012 0.395 0.030 0.9773 -- 
Hispanic -0.451 0.542 -0.830 0.4067 0.637 
Black -1.388 0.732 -1.900 0.0596 0.250 
SpeEd 1.260 0.577 2.190 0.0302 3.526 
FRL 0.422 0.334 1.260 0.2081 1.526 
ELL -0.434 0.684 -0.630 0.5267 0.648 
GRD8ELA 0.002 0.009 0.250 0.801 | 1.002 
GRD8Math 0.021 0.008 2.560 0.0113 1.021 
OAGRD9 -0.497 0.292 -1.700 0.0908 0.609 
GRD9ICRD -0.063 0.047 -1.340 0.1826 0.939 
GRD9ADA 0.014 0.013 1.080 0.2802 1.014 
AVGGRD8ELA 0.000 0.027 0.000 0.9964 1.000 
PFRL -0.029 0.034 -0.830 0.4067 0.972 
TRT 0.056 0.704 0.080 0.9371 1.057 

Random Component Estimate | SE 
Intercept 0.264 0.380 


Table D.61. HGLM results of the enrollment vs. discharge status for male students, 
2011-13 (final model) 


Fixed Effects Estimate | p-value | Odds Ratio 
Intercept 0.083 0.283 0.290 0.7748 -- 
Black -0.772 0.468 -1.650 0.1003 0.462 
SpeEd 1.197 0.543 2.200 0.0289 3.311 
GRD8Math 0.019 0.007 2.740 0.0067 1.019 
OAGRD9 -0.447 0.280 -1.600 0.1125 0.639 
TRT -0.085 0.484 -0.180 0.8610 0.919 

Random Component Estimate | SE 
Intercept 0.214 0.279 


Table D.62. HGLM results of the enrollment vs. discharge status for Hispanic male 
students, 2011-13 (full model) 


Fixed Effects | Estimate | t-ratio p-value Odds Ratio 
Intercept 0.148 0.476 0.310 0.7631 -- 
SpeEd 1.472 0.646 2.280 0.0245 4.357 
FRL 0.508 0.368 1.380 0.1698 1.662 
ELL -0.498 0.692 -0.720 0.4728 0.608 
GRD8ELA 0.003 0.010 0.280 0.7823 1.003 
GRD8Math 0.018 0.009 1.990 0.0482 1.018 
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Fixed Effects Estimate | t-ratio p-value Odds Ratio 
OAGRD9 -0.584 0.335 -1.750 0.0832 0.557 
GRD9ICRD -0.047 0.053 -0.890 0.3731 0.954 
GRD9ADA 0.006 0.015 0.420 0.6788 1.006 
AVGGRD8ELA 0.002 0.032 0.050 0.9587 1.002 
PFRL 0.003 0.042 0.070 0.9472 1.003 
TRT 0.003 0.824 0.000 0.9975 1.003 

Random Component Estimate | SE 
Intercept 0.339 0.695 


Table D.63. HGLM results of the enrollment vs. discharge status for Hispanic male 
students, 2011-13 (final model) 


Fixed Effects | Estimate | t-ratio | Odds Ratio 
Intercept 0.025 0.283 0.090 0.9325 -- 
SpeEd 1.257 0.611 2.060 0.0417 3.515 
FRL 0.531 0.354 1.500 0.1361 1.701 
GRD8Math 0.016 0.007 2.210 0.0287 1.016 
OAGRD9 -0.525 0.320 -1.640 0.1036 0.591 
TRT 0.103 0.418 0.250 0.8053 1.109 

Random Component Estimate | SE 
Intercept 0.073 0.287 


Table D.64. HGLM results of the enrollment vs. discharge status for students with 
less than I 1 credits at admission, 201 1-13 (full model) 


Fixed Effects | Estimate | t-ratio p-value | Odds Ratio 
Intercept 0.211 0.407 0.520 0.6167 -- 
Male -0.314 0.335 -0.940 0.3507 0.731 
Hispanic -0.321 0.497 -0.650 0.5186 0.725 
Black -0.065 0.673 -0.100 0.9234 0.937 
SpeEd 1.623 0.719 2.260 0.0254 5.066 
FRL 0.492 0.343 1.430 0.1537 1.636 
ELL 0.621 0.848 0.730 0.4648 1.861 
GRD8ELA 0.010 0.010 1.050 0.2975 1.010 
GRD8Math 0.016 0.009 1.720 0.0884 1.016 
OAGRD9 -0.408 0.320 -1.280 0.2041 0.665 
GRD9ICRD -0.027 0.072 -0.380 0.7052 0.973 
GRD9ADA 0.020 0.012 1.720 0.0876 1.020 
AVGGRD8ELA 0.026 0.029 0.920 0.3578 1.027 
PFRL -0.015 0.035 -0.430 0.6692 0.985 
TRT -0.382 0.742 -0.510 0.6080 0.683 

Random Component Estimate | SE 
Intercept 0.239 0.317 
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Table D.65. HGLM results of the enrollment vs. discharge status for students with 
less than I 1 credits at admission, 201 1-13 (final model) 


Fixed Effects Estimate | SE | t-ratio p-value Odds Ratio 
Intercept -0.098 0.278 -0.350 0.7301 -- 
SpeEd 1.168 0.662 1.760 0.0795 3.217 
FRL 0.469 0.330 1.420 0.1565 1.599 
GRD8Math 0.015 0.008 1.890 0.0605 1.016 
GRD9ADA 0.020 0.010 2.140 0.0341 1.021 
TRT 0.183 0.469 0.390 0.6970 1.201 

Random Component Estimate | SE 
Intercept 0.178 0.229 


Table D.66. HGLM results of the graduation vs. discharge status for female 
students, 2011-13 (full model) 


Fixed Effects | Estimate | SE t-ratio p-value Odds Ratio 
Intercept -0.220 0.355 -0.620 0.5503 -- 
Hispanic 0.390 0.397 0.980 0.3268 1.477 
Black -0.473 0.612 -0.770 0.4406 0.623 
SpeEd 0.112 0.716 0.160 0.8755 1.119 
FRL 0.167 0.302 0.550 0.5802 1.182 
ELL -0.919 0.805 -1.140 0.2549 0.399 
GRD8ELA -0.012 0.008 -1.450 0.1482 0.989 
GRD8Math 0.009 0.008 1.150 0.2512 1.009 
OAGRD9 0.065 0.327 0.200 0.8436 1.067 
GRD9ICRD 0.081 0.044 1.810 0.0713 1.084 
GRD9ADA 0.015 0.012 1.270 0.2047 1.015 
AVGGRD8ELA -0.048 0.029 -1.650 0.1011 0.953 
PFRL 0.048 0.036 1.330 0.1838 1.049 
TRT 1.302 0.670 1.940 0.0533 3.678 

Random Component Estimate | SE 
Intercept 0.149 0.297 


Table D.67. HGLM results of the graduation vs. discharge status for female 
students, 2011-13 (final model) 


Fixed Effects | Estimate | p-value | Odds Ratio 
Intercept -0.199 0.302 -0.660 0.5255 -- 
GRD9ICRD 0.094 0.035 2.720 0.0071 1.099 
AVGGRD8ELA -0.045 0.026 -1.730 0.0848 0.956 
PFRL 0.051 0.032 1.600 0.1117 1.052 
TRT 1.228 0.560 2.190 0.0292 3.415 

Random Component Estimate | SE 
Intercept 0.050 0.180 
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Table D.68. Logistic regression results of the graduation vs. discharge status for 
male students, 201 1-13 (full model) 


p-value Odds Ratio 


Wald Chi- | 


Predictors Estimate | SE 
Square 


Intercept 0.159 

Hispanic -0.573 0.453 1.599 0.2061 0.564 
Black -0.285 0.556 0.262 0.6085 0.752 
SpeEd 0.619 0.572 1.174 0.2785 1.858 
FRL 0.205 0.297 0.476 0.4903 1.228 
ELL -0.171 0.658 0.068 0.7949 0.843 
GRD8ELA 0.000 0.007 0.005 0.9444 1.000 
GRD8Math 0.002 0.007 0.140 0.708 | 1.002 
OAGRD9Y -0.023 0.267 0.007 0.9319 0.977 
GRDICRD -0.006 0.042 0.021 0.8836 0.994 
GRD9IADA 0.021 0.011 3.607 0.0575 1.021 
AVGGRD8ELA -0.020 0.019 1.114 0.2913 0.980 
PFRL -0.020 0.025 0.598 0.4394 0.980 
TRT 0.243 0.421 0.333 0.5640 1.275 


Table D.69. Logistic regression results of the graduation vs. discharge status for 
male students, 201 1-13 (final model) 


Wald Chi- 


Predictors Estimate | SE p-value Odds Ratio 
Square 
Intercept 0.296 0.200 2.202 0.1378 -- 
GRD9ADA 0.020 0.009 5.156 0.0232 1.020 
TRT -0.145 0.275 0.276 0.599 | 0.865 


Table D.70. Logistic regression results of the graduation vs. discharge status for 
Hispanic male students, 201 1-13 (full model) 


p-value Odds Ratio 


Wald Chi- | 


Predictors Estimate 
Square 


Intercept 

SpeEd 0.582 0.653 0.794 0.3731 1.789 
FRL 0.183 0.365 0.253 0.6151 1.201 
ELL -0.002 0.664 0.000 0.9974 0.998 
GRD8ELA 0.007 0.009 0.642 0.4231 1.007 
GRD8Math -0.004 0.008 0.225 0.6353 0.996 
OAGRD9 0.011 0.318 0.001 0.9719 1.011 
GRD9ICRD 0.008 0.049 0.025 0.8752 1.008 
GRD9ADA 0.009 0.013 0.515 0.4729 1.009 
AVGGRD8ELA -0.047 0.025 3.717 0.0539 0.954 
PFRL 0.004 0.035 0.014 0.9059 1.004 
TRT 0.703 0.509 1.910 0.1669 2.019 
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Table D.71. Logistic regression results of the graduation vs. discharge status for 
Hispanic male students, 201 1-13 (final model) 


Wald Chi- 


Predictors Estimate p-value Odds Ratio 
Square 


Intercept 
AVGGRD8ELA -0.045 0.022 4.292 0.0383 0.956 
TRT 0.637 0.488 1.710 0.1910 1.892 


Table D.72. HGLM results of the graduation vs. discharge status for students with 
less than || credits at admission, 201 1-13 (full model) 


Intercept -0.648 0.411 -1.580 0.1658 -- 
Male -0.206 0.366 -0.560 0.5752 0.814 
Hispanic 0.167 0.507 0.330 0.7418 1.182 
Black -0.561 0.740 -0.760 0.449 | 0.570 
SpeEd 1.785 1.184 1.510 0.1338 5.957 
FRL 0.414 0.366 1.130 0.2600 1.512 
ELL -0.497 0.941 -0.530 0.5984 0.608 
GRD8ELA -0.006 0.010 -0.640 0.5262 0.994 
GRD8Math -0.001 0.008 -0.170 0.8613 0.999 
OAGRD9 -0.114 0.358 -0.320 0.7505 0.892 
GRDICRD 0.021 0.075 0.270 0.7844 1.021 
GRD9IADA 0.007 0.011 0.670 0.5056 1.007 
AVGGRD8ELA -0.036 0.029 -1.250 0.2141 0.964 
PFRL 0.049 0.042 1.180 0.2417 1.050 
TRT 0.740 0.654 1.130 0.2600 2.095 
Random Component Estimate | SE 
Intercept 0.037 0.140 


Table D.73. HGLM results of the graduation vs. discharge status for students with 
less than I 1 credits at admission, 201 1-13 (final model) 


Fixed Effects | Estimate | | p-value | Odds Ratio 
Intercept -0.350 0.251 -1.390 0.2009 -- 
SpeEd 1.972 1.113 1.770 0.0785 7.181 
TRT 0.215 0.335 0.640 0.5222 1.240 


Random Component Estimate | SE 
Intercept 0.005 0.095 
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Full model specifications and regression analysis results for the 2011-2014 graduation and 
enrollment outcomes are presented below.*+ Due to the multiple dimensions of the data 
structure in this study (i.e., students nested within schools), hierarchical generalized linear 
modeling (HGLM) was conducted to account for the clustering effect and control for multiple 
covariates at each level within the same analysis. In cases where the between-school variance 
was zero (and therefore the corresponding random component estimate did not show up), 
logistic regressions were used to re-analyze the same sets of data and provide the final results. 
Note that in each regression model, all covariates were grand-mean centered, except for the 
treatment indicator. 


Full Hierarchical Generalized Linear Regression Model 


The two-level logistic regression model is given in terms of the logits of probabilities of the 
selected outcome equal to 1, Le., 

Pr(¥, =1) 
hi = log 


The full multilevel model can be specified as follows: 
Level 1: Student level 


jj = A; + a, (Male, — Male..) + a, (Hispanic ,, — Hispanic ..) + a, (Black,, — Black..) 
+ a, (SpeEd,, — SpeEd..) + a, (PRL, — FRL..)+ a, (ELL, — ELL..) 
+a, ,(GRD8ELA ,, - GRD8ELA..) + a, ,(GRD8Math ,, - GRD8Math..) 
+ a, (OAGRD9,, - OAGRD9..) + @,(GRD9CRD ,, — GRD9CRD..) 
+a,,,(GRD9ADA , —GRD9ADA..) + 7; 


where 
@; tepresents the mean logit for school / adjusted for the student-level covariates; 


“4 Specifically, in addition to the treatment indicator ([RT), the following covariates were included in each full 
model: student gender (Male), race/ethnicity (Hispanic, Black), free/reduced price lunch eligibility (FRL), 
English language learner (ELL) and special education status (SpeEd), New York State Grade 8 English 
Language Arts and Math exam scores (GRD8ELA, GRD8Math), number of years over-age when first 
enrolled in Grade 9 (OAGRDS), credits earned and average daily attendance (ADA) in Grade 9 (GRD9ICRD, 
GRD9IADA), and percent low-income and average New York State Grade 8 ELA exam score at the school- 
level (PFRL, AVGGRD8ELA). 
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@,;—@,,; teptesent the regression coefficients for school /, associated with various 


ij 
student-level covariates; and 


1, teptesents the random error associated with student 7in school /. 


Level 2: School level 
Qj = Boo + By (AVGGRD8ELA = AVGGRD8ELA.) + {),(PFRL ; ~PFRL.) 


as f3(TRT ;) +p; 


Qj = Bo 
Qj = Bao 
3; = Pao 
Q,; = Bay 
as; = Bso 
Qs; Boo 
at, ; = Bry 
aj Bao 
Qj Boo 
Qo; = Boo 
Qj; = Bao 
Where 


Bo teptesents the mean logit for the comparison schools; 
Bo — Bor teptesent the regression coefficients associated with the school-level covariates; 
Bo, teptesents the regression coefficient associated with the treatment indicator — it 


quantifies the treatment impact (the difference in the log-odds-ratio associated with 
being a treatment subject, as opposed to a comparison subject); 

g } J 
Bio— Bri teptesent the common regression coefficients associated with various student- 


level covariates for each school; and 
Uy, teptesents the random error associated with school 7. 


Full Logistic Regression Model 


The logistic regression model is given in terms of the logits of probabilities of the selected 
outcome equal to 1, Le., 


Pr(¥, = 1) 
1—Pr(Y, =1) 
The full model can be specified as follows: 


= log 
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1, = By + B, (Male, — Male.) + £, (Hispanic , — Hispanic .) + 8, (Black, — Black.) 
+ B,(SpeEd, —SpeEd.) + 2, (FRL, — FRL.)+ 6, (ELL, — ELL.) 
+ B,(GRD8ELA , - GRD8ELA.) + £, (GRD8Math , — GRD8Math.) 
+ B, (OAGRD9, — OAGRD9.) + £,,(GRD9CRD , — GRD9CRD.) 
+ B, (GRD9ADA, — GRD9ADA.)+ f,,(AVGGRD8ELA , — AVGGRD8ELA.) 
+ B,,(PFRL, — PFRL.) + f,,(TRT,) + &, 


where 


Y, represents the selected outcome for subject 4 

7, represents the logits of Pr(Y, 7 1) 

[3 tepresents the mean logit for subject 7 adjusted for the covariates; 

B, — B,3 tepresent the logistic regression coefficients associated with various covariates 


for subject 4 

f,, tepresents the logistic regression coefficient associated with the treatment indicator 
— it quantifies the treatment impact (the difference in the log-odds-ratio associated with 
being a treatment subject, as opposed to a comparison subject); 


&, tepresents the random error associated with subject 7. 
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Table E.!. Summary of regression results for overall impacts on intended outcomes 
(2011-14) 


e sya 2 a O e Probab re) @ e e salle 
: Comparison Treatment Comparison Treatment 

Enrollment vs Discharge 1I55x2 0.265 0.297 0.263 0.309 0.135 0.6499 

Graduation vs Discharge 291 x2 0.512 0.622 0.514 0.630 0.287 0.0207 


Table E.2. Summary of regression results for subgroup analyses of enrollment vs. 
discharge status (201 1-14) 


beroun O obability o ~ F 
Comparison Treatment Comparison Treatment oid 

Females 154 0.256 0.382 0.271 0.394 0.339 0.4132 

Males 156 0.273 0.215 0.255 0.222 -0.110 0.6429 

Black Males 22 

Hispanic Males 116 0.250 0.200 0.220 0.211 -0.033 0.9113 

< I1 credits at admission 145 0.324 0.312 0.317 0.298 -0.055 0.845 | 


Table E.3. Summary of regression results for subgroup analyses of graduation vs. 


discharge status (201 1-14) 


bgroup p e 
Comparison Treatment Comparison Treatment 
Females 298 0.532 0.662 0.532 0.678 0.374 0.0489 
Males 284 0.489 0.584 0.494 0.582 0.216 0.1448 
Black Males 43 
Hispanic Males 210 0.471 0.557 0.471 0.557 0.208 0.2160 
< 11 credits at admission 208 0.356 0.525 0.356 0.526 0.423 0.0175 
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Table E.4. HGLM results of the enrollment vs. discharge status for all students, 
2011-14 (full model) 


Fixed Effects | Estimate | t-ratio p-value Odds Ratio 
Intercept -0.933 0.374 -2.490 0.0342 -- 
Male -0.401 0.271 -1.480 0.1399 0.670 
Hispanic -0.386 0.401 -0.960 0.3375 0.680 
Black 0.086 0.498 0.170 0.8628 1.090 
SpeEd -0.497 0.535 -0.930 0.3533 0.608 
FRL 0.304 0.279 1.090 0.2777 1.355 
ELL -0.505 0.752 -0.670 0.5023 0.604 
GRD8ELA -0.010 0.006 -1.580 0.1158 0.990 
GRD8Math -0.001 0.007 -0.200 0.8446 0.999 
OAGRD9 0.113 0.263 0.430 0.6667 1.120 
GRD9ICRD -0.003 0.038 -0.070 0.9446 0.997 
GRD9ADA 0.004 0.011 0.330 0.7398 1.004 
AVGGRD8ELA 0.017 0.026 0.670 0.5065 1.018 
PFRL -0.050 0.029 -1.680 0.0933 0.952 
TRT -0.022 0.704 -0.030 0.9752 0.978 
Random Component Estimate | SE 
Intercept 0.314 0.374 


Table E.5. HGLM results of the enrollment vs. discharge status for all students, 
2011-14 (final model) 


Fixed Effects Estimate t-ratio p-value Odds Ratio 
Intercept -1.029 0.267 -3.850 0.0032 -- 
Male -0.394 0.263 -1.500 0.1355 0.675 
GRD8ELA -0.008 0.005 -1.490 0.1370 0.992 
PFRL -0.038 0.022 -1.750 0.0807 0.963 
TRT 0.223 0.490 0.450 0.6499 1.249 
Random Component Estimate | SE 
Intercept 0.272 0.331 


Table E.6. HGLM results of the graduation vs. discharge status for all students, 


2011-14 (full model) 


Fixed Effects Estimate SE t-ratio p-value Odds Ratio 
Intercept 0.028 0.183 0.150 0.8804 -- 
Male -0.252 0.177 -1.420 0.1553 0.778 
Hispanic -0.081 0.268 -0.300 0.7622 0.922 
Black -0.001 0.340 0.000 0.9968 0.999 
SpeEd 0.275 0.324 0.850 0.3967 1.316 
FRL 0.073 0.175 0.420 0.6750 1.076 


& metis associates making a meaningful difference 


109 


RE-ENGAGING YOUTH FOR HIGH SCHOOL SUCCESS 


APPENDIX E: REGRESSION ANALYSIS RESULTS, 2011-14 
—————— 


Fixed Effects Estimate | t-ratio p-value Odds Ratio 
ELL -0.067 0.427 -0.160 0.8746 0.935 
GRD8ELA 0.001 0.005 0.180 0.8538 1.001 
GRD8Math 0.003 0.004 0.780 0.4368 1.003 
OAGRD9 -0.128 0.168 -0.760 0.4447 0.880 
GRD9ICRD 0.033 0.026 1.270 0.2043 1.034 
GRD9ADA 0.014 0.007 1.950 0.0521 1.014 
AVGGRD8ELA -0.003 0.014 -0.230 0.8216 0.997 
PFRL -0.010 0.017 -0.560 0.5731 0.991 
TRT 0.542 0.321 1.690 0.0921 1.720 

Random Component Estimate | SE 
Intercept 0.027 0.056 


Table E.7. HGLM results of the graduation vs. discharge status for all students, 
2011-14 (final model) 


Fixed Effects | Estimate | | Odds Ratio 
Intercept 0.058 0.130 0.440 0.6673 -- 
Male -0.274 0.172 -1.600 0.1110 0.760 
GRD9IADA 0.019 0.006 3.280 0.0011 1.019 
TRT 0.474 0.204 2.320 0.0207 1.606 
Random Component Estimate | SE 
Intercept 0.019 0.042 


Table E.8. HGLM results of the enrollment vs. discharge status for female students, 
2011-14 (full model) 


Fixed Effects | Estimate | p-value | Odds Ratio 
Intercept -0.982 0.607 -1.620 0.1445 -- 
Hispanic -0.296 0.560 -0.530 0.5974 0.743 

Black 0.415 0.665 0.620 0.5342 1.514 
SpeEd -0.848 0.847 -1.000 0.3186 0.428 
FRL 0.580 0.412 1.410 0.1615 1.786 
ELL -0.386 1.043 -0.370 0.7121 0.680 
GRD8ELA -0.006 0.011 -0.570 0.5706 0.994 

GRD8Math -0.012 0.011 -1.150 0.2516 0.988 
OAGRD9 0.187 0.399 0.470 0.6399 1.206 

GRD9ICRD 0.011 0.053 0.220 0.8294 1.012 

GRD9ADA 0.009 0.015 0.600 0.5482 1.009 

AVGGRD8ELA 0.009 0.043 0.200 0.8400 1.009 
PFRL -0.021 0.046 -0.450 0.6559 0.980 
TRT 0.433 1.161 0.370 0.7101 1.541 
Random Component Estimate | SE 
Intercept 0.913 0.955 
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Table E.9. HGLM results of the enrollment vs. discharge status for female students, 
2011-14 (final model) 


Fixed Effects Estimate | SE t-ratio p-value | Odds Ratio 
Intercept -0.987 0.374 -2.640 0.0246 -- 
FRL 0.633 0.369 1.720 0.0884 1.883 
TRT 0.559 0.681 0.820 0.4132 1.749 
Random Component Estimate | SE 
Intercept 0.530 0.538 


Table E.10. Logistic regression results of the enrollment vs. discharge status for 
male students, 201 1-14 (full model) 


Predictors 


Estimate 


Odds Ratio 


Intercept -1.037 : 
Hispanic -0.88 | 0.614 2.056 0.1516 0.415 
Black -0.854 0.850 1.009 0.3150 0.426 
SpeEd -0.292 0.781 0.140 0.7082 0.747 
FRL 0.250 0.434 0.333 0.5637 1.285 
ELL -1.589 1.434 1.229 0.2676 0.204 
GRD8ELA -0.015 0.009 3.158 0.0756 0.985 
GRD8Math 0.010 0.010 0.998 0.3177 1.010 
OAGRD9 -0.037 0.397 0.009 0.9252 0.963 
GRD9ICRD -0.009 0.059 0.021 0.8854 0.991 
GRD9IADA -0.01 1 0.016 0.414 0.5200 0.989 
AVGGRD8ELA 0.017 0.027 0.405 0.5247 1.018 
PFRL -0.072 0.035 4.37] 0.0366 0.930 
TRT -0.371 0.619 0.360 0.5484 0.690 


Table E.I 1. Logistic regression results of the enrollment vs. 
male students, 2011-14 (final model) 


discharge status for 


Predictors Estimate | bi ceeds | p-value Odds Ratio 
Square 
Intercept -1.073 0.273 15.460 <.0001 -- 
PFRL -0.049 0.027 3.362 0.0667 0.953 
TRT -0.182 0.391 0.215 0.6429 0.834 
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Table E.12. Logistic regression results of the enrollment vs. discharge status for 
Hispanic male students, 201 1-14 (full model) 


Predictors Estimate p-value Odds Ratio 


Wald Chi- | 


Square 


Intercept 
SpeEd 0.245 0.842 0.085 0.7708 1.278 
FRL 0.560 0.507 1.221 0.2692 1.750 
ELL -1.862 1.540 1.461 0.2268 0.155 
GRD8ELA -0.017 0.010 2.875 0.0900 0.983 
GRD8Math 0.015 0.012 1.717 0.1901 1.016 
OAGRD9Y 0.372 0.481 0.598 0.4395 1.451 
GRD9ICRD -0.014 0.069 0.044 0.8335 0.986 
GRD9ADA -0.005 0.020 0.054 0.8165 0.995 
AVGGRD8ELA 0.025 0.034 0.552 0.4575 1.025 
PFRL -0.091 0.044 4.333 0.0374 0.913 
TRT -0.383 0.713 0.289 0.5907 0.682 


Table E.13. Logistic regression results of the enrollment vs. discharge status for 
Hispanic male students, 2011-14 (final model) 


Wald Chi- 


Predictors Estimate p-value Odds Ratio 
Square 


Intercept 
PFRL -0.069 0.036 3.598 0.0578 0.934 
TRT -0.054 0.483 0.012 0.9113 0.948 


Table E.14. HGLM results of the enrollment vs. discharge status for students with 
less than I 1 credits at admission, 201 1-14 (full model) 


Fixed Effects Estimate | t-ratio | p-value Odds Ratio 
Intercept 

Male 0.042 0.399 0.100 0.9172 1.042 
Hispanic -0.449 0.582 -0.770 0.4420 0.638 
Black 0.473 0.721 0.660 0.5135 1.604 
SpeEd 0.859 0.944 0.910 0.3646 2.361 
FRL 0.740 0.435 1.700 0.0917 2.095 
ELL 0.115 1.199 0.100 0.9240 1.121 
GRD8ELA 0.000 0.012 -0.020 0.9802 1.000 
GRD8Math 0.019 0.012 1.500 0.1374 1.019 
OAGRD9 0.357 0.405 0.880 0.3797 1.429 
GRD9ICRD -0.012 0.086 -0.140 0.8874 0.988 
GRD9ADA 0.038 0.016 2.280 0.0241 1.038 
AVGGRD8ELA 0.018 0.038 0.480 0.6316 1.018 
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Fixed Effects t-ratio | p-value Odds Ratio 
PFRL -0.004 0.044 -0.100 0.9234 0.996 
TRT -0.190 1.010 -0.190 0.8510 0.827 


Random Component 


Intercept 


0.768 


Table E.15. HGLM results of the enrollment vs. discharge status for students with 
less than I 1 credits at admission, 2011-14 (final model) 


Fixed Effects p-value Odds Ratio 
Intercept -0.768 0.296 -2.590 0.0250 -- 

FRL 0.788 0.395 1.990 0.0484 2.198 
GRD8Math 0.015 0.010 1.410 0.1611 1.015 
GRD9ADA 0.031 0.012 2.510 0.0132 1.032 

TRT -0.090 0.462 -0.200 0.845 | 0.914 

Intercept 0.110 0.294 


Table E.16. HGLM results of the graduation vs. discharge status for female students, 
2011-14 (full model) 


Fixed Effects Estimate | t-ratio p-value Odds Ratio 
Intercept 0.178 0.295 0.600 0.5618 -- 
Hispanic 0.213 0.365 0.580 0.5592 1.238 

Black -0.264 0.484 -0.550 0.5853 0.768 
SpeEd -0.288 0.497 -0.580 0.5632 0.750 
FRL 0.212 0.262 0.810 0.4199 1.236 
ELL -0.470 0.706 -0.670 0.5065 0.625 
GRD8ELA -0.002 0.007 -0.300 0.7652 0.998 
GRD8Math -0.002 0.006 -0.320 0.7524 0.998 
OAGRD9 -0.043 0.271 -0.160 0.8756 0.958 
GRD9ICRD 0.084 0.039 2.160 0.0314 1.087 
GRD9ADA 0.015 0.010 1.450 0.1476 1.015 
AVGGRD8ELA 0.000 0.023 0.020 0.9848 1.000 
PFRL -0.007 0.026 -0.260 0.7923 0.993 
TRT 0.574 0.562 1.020 0.3085 1.775 
Random Component Estimate | SE 
Intercept 0.154 0.180 
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Table E.17. HGLM results of the graduation vs. discharge status for female students, 
2011-14 (final model) 


Fixed Effects | Estimate | t-ratio p-value | Odds Ratio 
Intercept 0.129 0.186 0.690 0.5029 -- 
GRD9ICRD 0.071 0.035 2.010 0.0458 1.073 
GRD9ADA 0.014 0.010 1.480 0.1396 1.014 
TRT 0.618 0.312 1.980 0.0489 1.854 
Random Component Estimate | SE 
Intercept 0.056 0.093 


Table E.18. Logistic regression results of the graduation vs. discharge status for 
male students, 2011-14 (full model) 


Predictors | Estimate | SE reel | Odds Ratio 
Intercept -0.094 0.233 0.164 0.6853 -- 
Hispanic -0.494 0.422 1.365 0.2427 0.610 

Black -0.047 0.509 0.008 0.9272 0.955 
SpeEd 0.771 0.451 2.926 0.0872 2.163 
FRL -0.085 0.251 0.114 0.7352 0.919 
ELL 0.179 0.549 0.107 0.7441 1.196 
GRD8ELA 0.003 0.007 0.176 0.6748 1.003 
GRD8Math 0.008 0.006 2.095 0.1478 1.008 
OAGRD9Y -0.228 0.222 1.061 0.3029 0.796 
GRD9ICRD -0.009 0.038 0.054 0.8165 0.991 
GRDIADA 0.014 0.010 1.884 0.1699 1.014 
AVGGRD8ELA -0.009 0.018 0.247 0.6191 0.991 
PFRL -0.019 0.024 0.629 0.4277 0.981 
TRT 0.494 0.377 1.718 0.1899 1.639 


Table E.19. Logistic regression results of the graduation vs. discharge status for 
male students, 2011-14 (final model) 


p-value Odds Ratio 


Predictors Estimate 
Square 


Wald Chi- | 


Intercept 
SpeEd 0.680 0.424 2.564 0.1093 1.973 
GRD8Math 0.008 0.005 2.914 0.0878 1.008 
GRD9ADA 0.013 0.008 2.582 0.1081 1.013 
TRT 0.356 0.244 2.126 0.1448 1.428 
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Table E.20. Logistic regression results of the graduation vs. discharge status for 
Hispanic male students, 201 1-14 (full model) 


p-value Odds Ratio 


Wald Chi- | 


Predictors Estimate 
Square 


Intercept 
SpeEd 0.671 0.494 1.846 0.1743 1.957 
FRL -0.212 0.292 0.527 0.4680 0.809 
ELL 0.244 0.550 0.197 0.6572 1.276 
GRD8ELA 0.008 0.008 1.138 0.2860 1.008 
GRD8Math 0.003 0.007 0.256 0.6132 1.003 
OAGRD9Y -0.168 0.250 0.454 0.5006 0.845 
GRD9ICRD 0.007 0.043 0.023 0.8795 1.007 
GRD9ADA 0.008 0.011 0.464 0.4960 1.008 
AVGGRD8ELA -0.014 0.019 0.539 0.4629 0.986 
PFRL 0.001 0.030 0.001 0.9773 1.001 
TRT 0.561 0.425 1.744 0.1866 1.752 


Table E.21. Logistic regression results of the graduation vs. discharge status for 
Hispanic male students, 201 1-14 (final model) 


Wald Chi- 


Predictors Estimate p-value Odds Ratio 
Square 


Intercept 
TRT 0.343 0.277 1.531 0.2160 1.409 


Table E.22. HGLM results of the graduation vs. discharge status for students with 
less than I | credits at admission, 201 1-14 (full model) 


Fixed Effects Estimate | p-value Odds Ratio 
Intercept -0.824 0.335 -2.460 0.0363 -- 

Male -0.037 0.303 -0.120 0.903 | 0.964 
Hispanic -0.057 0.451 -0.130 0.9003 0.945 
Black -0.696 0.641 -1.090 0.2790 0.498 
SpeEd 0.404 0.662 0.610 0.5423 1.498 
FRL -0.220 0.306 -0.720 0.4735 0.803 
ELL -0.269 0.716 -0.380 0.7074 0.764 
GRD8ELA 0.005 0.008 0.580 0.5596 1.005 
GRD8Math -0.008 0.008 -1.000 0.3179 0.992 
OAGRD9 0.107 0.307 0.350 0.7292 1.112 
GRD9ICRD 0.042 0.063 0.650 0.5136 1.042 
GRD9ADA 0.005 0.010 0.560 0.5774 1.005 
AVGGRD8ELA -0.020 0.023 -0.880 0.3800 0.980 
PFRL 0.004 0.028 0.150 0.8774 1.004 
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Fixed Effects | Estimate | SE t-ratio p-value Odds Ratio 


A 


Random Component | Estimate 


Intercept 


Table E.23. HGLM results of the graduation vs. discharge status for students with 
less than I 1 credits at admission, 2011-14 (final model) 


Fixed Effects i t-ratio p-value Odds Ratio 
Intercept -0.593 0.222 -2.680 0.0215 -- 
TRT 0.698 0.291 2.400 0.0175 2.010 


Random Component 


Intercept 0.003 0.072 
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The Developmental Assets Profile (DAP), developed by the Search Institute, has been used to 
evaluate the impact of youth development programs and other school-based interventions 
(Haggerty et al., 2011). It includes 58 items that measure eight asset categories, which are 
organized into an External context (support, empowerment, boundaries and expectations, and 
constructive use of time) and an Internal context (commitment to learning, positive values, social 
competencies, and positive identity). The items can also be regrouped into five scales that reflect 
the contexts of students’ lives: Personal, Social, Family, School, and Community. Figure F.1 lists 
the categories of the developmental assets. Figure F.2 describes the categories and how they may 
be interpreted. 


The Total assets score is the most global index that can be derived from the DAP. High total 
assets scores, in the excellent range, are related to positive outcomes, such as academic success, 
leadership, and protection against negative behaviors. Low Total assets scores are associated with 
negative behaviors, such as substance use/abuse, academic problems, peer conflict, and 
antisocial or violent behaviors. 


Sub-scale scores are provided on a scale of 0-30, and the total DAP (the sum of the Internal and 
External sub-scale scores) is reported on a scale of 0-60. Score ranges based on positive youth 
development theory were established that describe at-risk, vulnerable, healthy, and thriving 
youth. Although those ranges were set theoretically and not empirically, research has shown that 
the four levels meaningfully differentiate youth at differing levels of well-being (Search Institute, 
2005; Scales, 2011; Scales et al., 2012). Search Institute data indicate that the average young 
person experiences only 20 of the 40 assets (Benson, 2011). 


Internal consistency for both the Internal/External assets and Social context areas within the 
DAP are good (Cronbach’s O for assets measure = 0.81; Cronbach’s Q for social context area = 
0.88). Further, Haggerty et al. (2011) indicated that the instrument has evidence of criterion and 
convergent validity. 


The DAP has been found to be highly reliable and valid with U.S. samples, with internal 
consistency reliabilities in the 0.70s-0.80s, and stability reliabilities as measured by intraclass 
cottelation coefficients in the 0.50s-0.80s, and most of its sub-scales have been found to be 
acceptably reliable and valid with international samples as well (Scales, 2011; Scales et al., 2012; 
Search Institute, 2005). 


The DAP also has been correlated with indicators of college and career readiness (CCR), defined 
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as: 1) the frequency with which students experience “key cognitive strategies” (e.g., problem 
solving, reasoning, communication) in the core subjects of Math, English, and Science; 2) 
students’ time management and study habits; 3) students’ perception of the academic rigor of 
their core classes (the three subjects above, plus Social Studies); 4) the degree to which students 
are engaged in researching colleges; and 5) the level of support students get from school and 
family in learning about colleges and how to apply. All of these CCR outcome scales have score 
ranges from 10-40 except College Research, which ranges from 10-30, and all have acceptable 
internal consistency reliability in the 0.70s-0.90s in a Dallas Independent School District spring 
2011 sample (Search Institute, 2012). 


Figure F.1. The 40 Developmental Assets 


External Assets Internal Assets 


Developmental Assets 


Category Developmental Assets 


Support |. Family support Commitment | 21. Achievement motivation 
2. Positive family communication fe Teste 22. School engagement 
3. Other adult relationships 23. Homework 
4. Caring neighborhood 24. Bonding to school 
5. Caring school climate 25. Reading for pleasure 
6. Parent involvement in Positive 26. Caring 
schooling Values 
Empowerment | 7. Community values youth 27. Equality and social justice 
8. Youth as resources 28. Integrity 
9. Service to others 29. Honesty 
10. Safety 30. Responsibility 
Boundaries and | ||. Family boundaries 31. Restraint 
pe igs 12. School boundaries Social 32. Planning and decision making 


Competencies 


13. Neighborhood boundaries 33. Interpersonal competence 


14. Adult role models 34. Cultural competence 

15. Positive peer influence 35. Resistance skills 

16. High expectations 36. Peaceful conflict resolution 
Constructive 17. Creative activities Positive 37. Personal power 
Econ 18. Youth programs Enel 38. Self-esteem 

19. Religious community 39. Sense of purpose 

20. Time at home 40. Positive view of personal 

future 
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Figure F.2. DAP score interpretation by category and subscale 


Scoring Interpretation 


Category and Subscale 


Description 


Parent-adolescent communication, family 
support, as well as caring, encouragement, 


Lack of or infrequent support; 
increased risk for a range of negative 


Excellent 


High academic performance and 


uppor : ‘ i an 
Support and support outside the family from the outcomes, particularly school thriving among males and females 
neighborhood, school, and community problems 
A general feeling of safety across many 3 : a Reduced risk of depression, suicidal 
‘ Increased risk for depression, suicidal ie . 
Empowerment contexts; feeling valued, useful, and feat an end alanine and self-injurious behaviors, and 
a respected by others : violence 
£ P | eed : f Associated with an increased risk of 
io y ae Suppers Sale ne ey i . depression, suicide, and antisocial : 
1 Boundaries and settings; rules and consequences in a variety gtevinparnena dil veudke iGrensea Strongly and consistently related to 
Expectations of settings; and role models among friends, risk oF drug ibs ie prablenis high academic achievement 
family, and outside the family 
among males 
Involvement in extracurricular activities in ; : F ees : 
c vel ff - (1) religi a) Increased risk for alcohol, tobacco, Associated with thriving; high degree 
Ae eee Oneco Jour sess: (i) eelisious- or spinal and drug problems as well as school of reported extracurricular 
i 2 rts, clubs, or other groups; (3) BP P 
of Time Cyspere , dita roblems among males involvement 
; and (4) family lif P 5 
creative arts; and (4) family life 
ae : Poor academic performance, : ae 
Motivation and rewards related to learning ee nid inereasedinice: High degree of reported motivation to 
Commitment to and active engagement in learning, both tied af disnaucand schasl eieked learn and active engagement in learning 
Learning directly to school and extending outside of ee ne wallasciiieodal both in and out of school; strongly 
school P ie related to academic achievement 
behavior among males 
Bisrecaal wieeeuel ae henekbicintcenk Lack of personal values, which is Benefits to current and future 
Eecaaneibiliiicand reseninicas Rad Baal related to increased risk for alcohol, decision-making skills; strong 
= Positive Values ee orale ecrhin fon saualin and 8 tobacco, and drug use; school association with thriving and increased 
iS secisliusine 8 ee problems; violence; and antisocial likelihood of significant community 
@ } behaviors service and volunteerism 
< 


Social 
Competencies 


Planning and decision making, cultural 
competence, and social skills involving the 
ability to build friendships, resist negative 
peer pressure, and resolve conflicts 
peacefully 


Increased risk behaviors, including 
peer conflict, antisocial behavior, and 
violence 


Rich set of characteristics that 
promote thriving, particularly with 
diversity and leadership, and reduced 
risk of negative youth outcomes 


Positive Identity 
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Adolescent’s emerging identity, including 
self-esteem, internal locus of control, 
optimism, and sense of purpose in life 


Increased risk for anxiety, depression, 
suicide, or self-injurious behavior 


Increased psychological resilience and 
reduced risk for psychological distress, 
such as anxiety and depression 
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A total of 119 new students entering the GSS transfer schools for the first time were asked to 
complete the DAP prior to the start of the 2012-13 school year, essentially representing baseline 
assets levels. Relatively small percentages of students were considered af risk, but the largest 
group of students was at the va/nerable level. About one-third (34%) scored at the healthy assets 
level, and /hriving students accounted for 17 percent. (See Figure F.3). The levels reported by the 
Search Institute for its Dallas sample (see Appendix H), with 13 percent considered at risk and 
34 percent healthy, were within the range of the levels of the GSS students. Compared to the GSS 
students, a slightly smaller percentage of Dallas students were considered “hriving (11% to 17%) 
and a slightly larger percentage vulnerable (42% to 39%). 


Figure F.3: Level of Developmental Assets, fall 2012 GSS incoming students and 
Search Institute Dallas sample 


100% ~ 
17% 11% 
80% 
34% 
34% 
60% - 
Thriving 
40% 
Healthy 
20% - = Vulnerable 
m At-Risk 
0% = 
GSS (N = 119) Dallas sample (N = 20,117) 


Table F.1 presents the mean fall 2012 and spring 2013 scale scores for the matched GSS transfer 
school students (those students with both a pre- and post- DAP score) as well as the 
corresponding means for the Dallas sample. For the group of GSS students with longitudinal 
data (N=75), mean DAP scale scores at baseline look somewhat similar to the Dallas sample, 
with most notable differences among the External subscales of Support, Boundaries and 
Expectations, and Constructive Use of Time. Although post score differences for the matched 
GSS group appear minimal, there is a statistically significant gain in Constructive Use of Time. 
This is notable, as the post-score in Constructive Use of Time is a closer match to the Dallas 
sample than the pre-score. 
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Table F.1. Mean DAP Scores, GSS transfer school students, fall 2012 and spring 


2013, and Dallas sample 


Dallas students (N = 75) 
sample Fall 2012 Spring 2013 
atego bscale (N = 20,117) (Pre) (Post) 
ee a en 2053 12253 [2181 
‘ a A le 
f FO tg Od 
16.86 14.84 16.63* 


Internal 


Social contexts 


Community 


GSS transfer school 


21.23 


es ee az 

on a 

en a 
22.42 | 22.22 


External 19.56 20.39 20.72 
Internal 20.82 21.07 21.05 
Total DAP 40.39 41.46 41.77 


*The difference is statistically significant at the .05 level; effect size = .22 


Table F.2 focuses specifically on the pre- and post- responses of GSS students to the External 
assets items on the DAP. Large percentage point differences (i.e., greater than 10) between pre 
and post items include friends who set a good example for me (+11), involved in a religious group or activity 
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(+17), involved in a sport, club, or other group (+11) and involved in creative things such as music, theatre, or art (+15). Note that the latter three of 
these four items fall within the Constructive Use of Time subscale. 


Table F.2. Distribution of responses to DAP, External assets by subscale and item, fall 2012 and spring 2013 


Not at | Somewhat Vv Extremely Not at Somewhat Extremely 
ery or Very or espo 
all or or or almost all or or or almost 
. often ‘ often ery/ofte 
rarely sometimes always rarely sometimes always 2 
D ale and = a O 
Seek advice from my parents. 11% 27% 27% 36% 11% 23% 31% 36% 4 
Parent(s) who try to help me succeed. 3% 3% 21% 73% 3% 9% 28% 60% -7 
Good neighbors who care about me. 15% 25% 29% 31% 23% 25% 21% 31% -8 
5 A school that cares about kids and 
a encourages them. 1% 7% 37% 55% 4% 7% 31% 59% -3 
te Support from adults other than my parents. 3% 9% 41% 47% 3% 19% 28% 51% -9 
A family that gives me love and support. 1% 8% 28% 63% 0% 12% 31% 57% -3 
Parent(s) who are good at talking with me 
about things. 4% 20% 29% 47% 8% 17% 28% 47% -| 
Feel safe and secure at home. 0% 4% 28% 68% 0% 9% 28% 63% -5 
's Feel valued and appreciated by others. 5% 29% 40% 25% 3% 31% 32% 35% I 
E Feel safe at school. 1% 11% 44% 44% 3% 8% 31% 59% I 
3 Included in family tasks and decisions. 3% 21% 45% 31% 3% 21% 36% 40% 0 
& Given useful roles and responsibilities. 3% 23% 43% 31% 3% 16% 43% 39% 7 
A safe neighborhood. 3% 31% 35% 32% 5% 20% 36% 39% 8 
8 5 Friends who set good examples for me. 4% 36% 32% 28% 9% 20% 39% 32% 11 
Ss z 8 A school that gives students clear rules. 3% 9% 39% 49% 4% 9% 39% 48% -l 
ceo VY 
3 | Adults who are good role models for me. 0% 4% 43% 53% 3% 11% 33% 53% -9 
i Teachers who urge me to develop and 3% 12% 37% 48% 1% 5% 36% 57% 8 
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0 D g 20 g 
om 2012 to 
Not at | Somewhat Vv Extremely Not at Somewhat Extremely : 
ery or Very or espo 
all or or or almost all or or or almost 
P often 2 often ery/ofte 
rarely sometimes always rarely sometimes always 2 
D ale and = a fe) 
achieve. 
A family that provides me with clear rules. 1% 7% 39% 53% 5% 8% 40% 47% -5 
Parent(s) who urge me to do well in 
school. 3% 4% 25% 68% 3% 7% 28% 63% -3 
Neighbors who help watch out for me. 17% 24% 25% 33% 22% 24% 23% 31% -5 
A school that enforces rules fairly. 3% 9% 40% 48% 5% 13% 36% 45% -7 
A family that knows where | am and what | 
am doing. 1% 12% 28% 59% 4% 19% 29% 48% -9 
Involved in a religious group or activity. 47% 25% 13% 15% 37% 17% 19% 27% 17 
Yo 
Fa é Involved in a sport, club, or other group. 41% 12% 20% 27% 31% 12% 23% 35% 11 
5 S Involved in creative things such as music, 
5 » theatre, or art. 26% 25% 22% 26% 17% 19% 32% 32% 15 
OD Spending quality time at home with my 
parent(s). 4% 24% 24% 48% 10% 26% 30% 35% -7 


Likewise, Table F.3 focuses on the pre- and post- responses of GSS students to the Internal assets items on the DAP. Large 
percentage point differences (i.e., greater than 10) between pre and post items include enjoy reading or being read to (+10), care about school 
(-11), stay away from tobacco, alcohol, and other drugs (-12), serving others in my community (+11) and resolve conflicts without anyone getting hurt (-10). 
Unlike the differences observed for the External scale, two of the larger differences between pre and post on the Internal scale were 
negative, and none were concentrated within a specific subscale. 
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Table F.3. Distribution of responses to DAP, Internal assets by subscale and item, fall 2012 and spring 2013 


Internal assets 


Fall 2012 Spring 2013 Peneentige 
point change, 


Subscales and Items 2012 to 2013 


Not at | Somewhat Very Extremely Notat Somewhat Very Extremely [MMVay(oic-n 
all or or or oralmost all or or or ClaeVutesa + extremely/ 
rarely sometimes often always rarely sometimes often always almost always 


Enjoy reading or being read to. 12% 43% 24% 20% 15% 31% | 27% 28% 
Care about school. 1% 17% 36% 45% 1% 28% | 31% 40% 
Do my homework. 7% 34% 32% 27% 9% 32% | 29% 29% 
Enjoy learning. 4% 19% 47% 31% 4% 27% | 31% 39% 
Actively engaged in learning new things. 0% 13% 41% 45% 0% 17% | 47% 36% 


Commitment 
to Learning 


Encouraged to try things that might be good for me. 0% 11% 42% 47% 1% 10% | 47% 42% 
4% 35% 60% 7% | 41% 51% 


Eager to do well in school and other activities. 
Stand up for what | believe in. 

Stay away from tobacco, alcohol, and other drugs. 
Think it is important to help other people. 

Take responsibility for what | do. 

Tell the truth even when it is not easy. 

Helping to make my community a better place. 


Developing good health habits. 


Positive Values 


Encouraged to help others. 
Trying to help solve social problems. 
Developing respect for other people. 


Serving others in my community. 


2 § Avoid things that are dangerous or unhealthy. 
(J d 
9 a Building friendships with other people. 5% 24% 35% 35% 4% 26% | 34% 37% 0 
n 
3 Express my feelings in proper ways. 4% 29% 36% 31% 3% 29% | 36% 32% | 
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Internal assets 


Fall 2012 Spring 2013 Rencenuage 
point change, 
2012 to 2013 
Notat Somewhat Very Extremely Notat Somewhat Very Extremely (Mi 7leicu 


all or or or oralmost all or or or Clara + extremely/ 
rarely sometimes often always rarely sometimes often always almost always 


Subscales and Items 


0% 15% 37% 48% 0% 20% | 33% 47% 
Resist bad influences. 1% 23% 35% 41% 4% 27% 32% 37% 
Resolve conflicts without anyone getting hurt. 3% 15% 46% 37% 4% 23% | 37% 36% 
Accept people who are different from me. 0% 3% 31% 67% 1% 8% | 24% 66% 
Sensitive to the needs and feelings of others. 3% 24% 32% 41% 5% 20% | 36% 39% 


Plan ahead and make good choices. 


Feel in control of my life and future. 4 
Feel good about myself. 3 
Feel good about my future. 5 


Deal with frustration in positive ways. 3 


Positive Identity 


Overcome challenges in positive ways. 5 


Developing a sense of purpose in my life. 3 
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Appendix G: Regression Results for the 
Development Assets Profile 


While changes between pre- and post-items and scales provide some insight into socio- 
emotional changes students may go through during the school year, a bigger question is the 
extent to which student characteristics, school performance and behavior may be associated with 
DAP scale scores. To assess the relationship between select student characteristics and 
performance metrics and spring 2013 DAP scores, a series of multiple linear regressions were 
conducted for the same subgroup of individuals for whom longitudinal data were available 
(N=75). Data collected from both administrations of the DAP were merged with archival 
student data collected from the NYC DOE, internship participation data acquired from GSS, 
and responses to select spring 2013 student survey items related to engagement. Of note, 8 
grade achievement data were considered for inclusion as predictors, but were eliminated due to 
several cases missing data. In total, 14 variables were modeled as predictors of spring 2013 Total 
DAP scores as well as subscale scores for External, Internal, School Context and Constructive 
Use of Time. 


The complete list of predictor variables were: 
e Pre DAP Score 
e Grade Level in 2013 
@ Sex (O=Female, 1=Mak) 
e Race/Ethnicity 
oO Black vs other races (O=Other, 1=Biack) 
oO Hispanic vs other races (O=Orher, 1=Hispanic) 
e English Language Learner Status 2013 (0=Not ELL, 1=ELL) 
e Eligibility for Free/Reduced Price Lunch 2013 (0=Not EXgible, 1=Eligible) 
Special Education Status 2013 (0=Not Special Ed, 1=Special Ed) 
Average Daily Attendance 2013 
e Credits Taken in 2013 
e Credits Earned in 2013 
e Age at Start of SY 2012-13 
Had an Internship during SY 2012-13 (O=No, 1=Yes) 
Student Survey items (from spring 2013 administration) 
o Q3a. I led an activity (discussion group, service project) (O=No, 1=Yes) 
Oo Q3c. [helped plan a program activity or event (O=No, 1=Yes) 
o Q3g. I contributed solutions for a community problem (0=No, 1=Yes) 
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All regression models were conducted using a stepwise approach, wherein only those variables 
that predict outcomes with statistical significance were included in final models. The resulting 
models are presented below in Tables G.1 through G.5. Each table presents the predictor 
variable(s) included in the final predictive model, the variable’s standard deviation change 
associated with a full standard deviation change in the outcome (standardized beta, negative 
predictors denoted in red), and the proportion of variance explained (change in r?) by inclusion 
of the variable in the model. 


Overall, the resultant models had strong predictive value for spring 2013 DAP scores, with three 
of the five explaining over half (0.5) of the variance in the outcome (Total, Internal and 
Constructive Use of Time) and one explaining over 40% (0.433 — External) as measured by 
model 1? values. These four models all included fall 2012 DAP scores and affirmative responses 
to contributing solutions to a community problem. The three models explaining over half of the 
variance also included credits earned in 2013. The final model, explaining just under a third of 
the variance (0.304 — school context), only included the fall 2012 DAP score. 


It should also be noted that participants of other races appeared to be associated with higher 
levels of total assets than Black participants. As the same relationship was not observed for the 
other DAP subscales, this finding may be a statistical anomaly. However, the result may warrant 
some further investigation regarding possible differential socio-emotional effects or service 
delivery biases that may result in lower levels of assets for Black participants. 


Overall, the regression findings seem to confirm the relationship between academic progress—as 
measured by credits earned—and Total assets, Internal assets and assets related to Constructive 
Use of Time. Also notable is the apparent relationship between asset levels and contributions to 
community solutions for Total, Internal, External and Constructive Use of Time assets—a sense 
of community that may very well have been fostered by the “family” experience to which 
participants referred in focus groups. 


Table G.1. Model 1 — Predicted Post Total DAP Scale Score (N=70, 12=0.546) 


Variable Standardized B _—r? Change 
Pre DAP Total Scale Score 0.590 0.432 
Q3¢. I contributed solutions for a community problem 0.202 0.048 
Credits Earned in 2013 0.216 0.037 
Black vs Other Races -0.173 0.029 
Constant 5.648 
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Table G.2. Model 2 — Predicted Post DAP External Assets Scale Score 
(N=70, 1?=0.433) 


Variable Standardized B _—r? Change 

Pre DAP External Assets Scale Score 0.569 0.394 

Q3g¢. I contributed solutions for a community problem 0.205 0.039 | 
Constant 2.489 | 


Table G.3. Model 3 — Predicted Post DAP Internal Assets Scale Score 
(N=70, 12=0.528) 


Variable Standardized 6 =r? Change 
Pre DAP Internal Assets Scale Score 0.551 0.443 
Credits Earned in 2013 0.229 0.055 
Q3g¢. I contributed solutions for a community problem 0.239 0.050 
Constant F265 


Table G.4. Model 4 — Predicted Post DAP School Context Scale Score 
(N=70, r?=0.304) 


Variable Standardized 8B rr? Change 
Pre DAP School Context Scale Score O55] 0.304 
Constant 7.985 | | 


Table G.5. Model 5 — Predicted Post DAP Constructive Use of Time Scale Score 
(N=70, 1?=0.573) 


Variable Standardized 8B —r? Change 
Pre DAP Constructive Use of Time Scale Score 0.620 0.483 
Q3g. I contributed solutions for a community problem 0.250 0.063 
Credits Earned in 2013 0.163 0.026 
Constant -0.488 
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Appendix H: Search Institute Report on the Dallas 
Sample 


S h 615 First Avenue NE 
Ca¥rCc Minneapolis, MN 55413 
INSTITUTE 1.800.888.7828 
Discovering what kids need to succeed www.search-institute.org 
Search Institute Report to Good Shepherd Services, New York City: 
Supplemental Analysis of the Dallas Developmental Assets Profile Dataset 
Nov. 26, 2012 


Background 

Good Shepherd Services is a youth development and family services organization serving 
more than 20,000 vulnerable children and youth per year in high-poverty communities in 
the Bronx and Brooklyn, in New York City. They provide wrap-around services that 
focus on keeping young people connected with school and strengthening families and 
neighborhoods. GSS has retained Metis Associates, a research and evaluation firm, to 
evaluate the success of GSS transfer high schools in South and West Brooklyn. As part of 
the evaluation, GSS and Metis plan to include Search Institute’s Developmental Assets 
Profile (DAP). 


At the request of the W.T. Grant Foundation, Search Institute provided some initial 
consultation to the GSS team on the DAP. Subsequently, GSS requested Search Institute 
to conduct supplemental analysis of a large DAP dataset that could provide GSS and 
Metis with comparison data to utilize in the evaluation. 


Description of Sample 

GSS requested analysis on a sample that is comparable to its transfer samples in key 
demographics. Search Institute analyzed DAP data gathered in spring 2011 in the Dallas 
Independent School District, from more than 40,000 6-12" grade students. This report 
focuses on the high school part of the sample that is of most interest to GSS. 


The high school sample included 20,241 students, of whom 68% are Hispanic, 24% 
African American, 6% white, and 3% other, mostly Asian, and among whom 85% are 
poor, as defined by their eligibility for free or reduced price meals. This compares with 
the GSS schools’ sample across two Brooklyn, New York schools that is 64%-73% 
Hispanic, 9%-21% African American, 3%-17% white, and 1%-9% other, mostly Asian, 


& metis associates making a meaningful difference 130 


RE-ENGAGING YOUTH FOR HIGH SCHOOL SUCCESS 
APPENDIX H: SEARCH INSTITUTE REPORT ON THE DALLAS SAMPLE 


and among whom 68%-80% are poor. Thus, the Dallas sample is demographically quite 
similar to the GSS samples in race/ethnicity and poverty level, and therefore provides an 
appropriate source of comparison." 


Research Questions 
Search Institute conducted analyses to answer two questions: 

1. What is the level of developmental assets among the Dallas high school students, 
specifically, among Hispanic and African American students, among girls as 
compared with boys, and, by individual grades within the high school sample? 

2. What is the relationship of the level of developmental assets to educational 
outcomes among these students? 


These questions were designed to provide GSS with guidance for the level of 
developmental assets they might realistically expect from their Brooklyn evaluation 
samples, and for the kind of relationship they might expect to see between assets and 
important educational outcomes. The latter is important because they key goal of the GSS 
transfer schools is to increase the school success of those students. If the level of assets is 
highly correlated with educational outcomes in the demographically similar Dallas 
sample, then the same relationships should be apparent in the GSS schools, suggesting 
the possible value of intentionally attempting to raise students’ assets levels. 


Analyses Conducted 

Below, we provide simple frequencies and cross-tabulations to answer the first question 
(specifically, we provide the mean scores for the overall DAP and its various sub-scales, 
and the percentage of Dallas students scoring at each of four levels of the DAP: at risk, 
vulnerable, healthy, and thriving). To answer the second question, we conducted a series 
of cross-tabulations and analyses of variance. These provide, specifically, the percentage 
of students who attain criterion levels of the educational outcomes (described below; 
criterion level was defined as being at or above the median score for a given outcome), 
by level of DAP score (i.e., at risk, vulnerable, healthy, and thriving), and the 
significance of differences in the means of those educational outcomes, by level of the 
overall DAP score. Analyses of variance are provided separately by race/ethnicity, grade, 
and gender. 


*° For the data reported by race/ethnicity, the “Other” race/ethnicity category was 
dropped, because the presence of multiple races/ethnicities in that category makes it 
difficult to clearly interpret the results. The final sample for those specific analyses was 
then 19,254 high school youth, 69% Hispanic, 25% Black, and 6% white. 
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Measures 

The Developmental Assets Profile is a 58-item survey that asks young people about the 
frequency or intensity with which they experience a variety of relationships, 
opportunities, values, skills, and self-perceptions—developmental assets—which studies 
of more than 3.5 million youth have shown concurrently and longitudinally predict 
numerous academic, psychological, socio-emotional, and behavioral outcomes, both in 
the U.S. and globally (Benson, Scales, & Syvertsen, 2011; Benson, Scales, 
Roehlkepartain, & Leffert, 2011; Scales, Roehlkepartain, & Fraher, 2012). Items reflect 
eight categories of developmental assets: Support, Empowerment, Boundaries & 
Expectations, and Constructive Use of Time are “external” strengths provided by adults 
and peers, and Commitment to Learning, Positive Values, Social Competencies, and 
Positive Identity are strengths young people gradually develop as they become more self- 
regulating. The items can be re-grouped to form five scales that reflect the various 
contexts of students’ lives: Personal, Social, Family, School, and Community. 


Students answer each item on a four-point scale: Not at All/Rarely, 
Somewhat/Sometimes, Very/Often, and Extremely/Almost Always. Sub-scale scores are 
provided on a scale of 0-30, and the total DAP (the sum of the Internal and External sub- 
scale scores) is reported on a scale of 0-60. Score ranges based on positive youth 
development theory were established that describe at-risk, vulnerable, healthy, and 
thriving youth. Although those ranges were set theoretically and not empirically, research 
has shown that the four levels meaningfully differentiate youth at differing levels of well- 
being (Search Institute, 2005; Scales, 2011; Scales et al., 2012). The DAP has been found 
to be highly reliable and valid with U.S. samples, with internal consistency reliabilities in 
the .70s-.80s, and stability reliabilities as measured by intraclass correlation coefficients 
in the .50s-.80s, and most of its sub-scales have been found to be acceptably reliable and 
valid with international samples as well (Scales, 2011; Scales et al., 2012; Search 
Institute, 2005). 


The educational outcomes examined in these analyses were used as indicators of “college 
and career readiness” (CCR): 1) the frequency with which students experience “key 
cognitive strategies” (e.g., problem solving, reasoning, communication) in the core 
subjects of Math, English, and Science; 2) students’ time management and study habits; 
3) students’ perception of the academic rigor of their core classes (the three subjects 
above, plus Social Studies); 4) the degree to which students are engaged in researching 
colleges; and 5) the level of support students get from school and family in learning about 
colleges and how to apply. All these CCR outcome scales have score ranges from 10-40 
except College Research, which ranges from 10-30, and all have acceptable internal 
consistency reliability in the .70s-.90s in the Dallas ISD spring 2011 sample. 
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Results 
A. Level of Developmental Assets 
Race/Ethnicity 


Tables 1 (percentages of students in each of the four levels of total DAP scores) and 2 
(mean scores on all 16 DAP scales) show that, by race/ethnicity, Black and white 
students reported roughly comparable levels of assets, with Hispanic students reporting 
the lowest levels of assets. 


By way of comparison, the Dallas sample had almost exactly the same total percentage of 
students in the combined at risk and vulnerable levels—55%—as did the original 2005, 
more white and suburban sample of high school students on which the DAP was field- 
tested (56%). Likewise, the total Dallas percentage that would be considered as 
experiencing adequate developmental nourishment by virtue of being in the combined 
healthy and thriving levels—45%—was not materially different from the percentage of 
healthy plus thriving students in that more white and suburban field test sample (44%). 


Table 2 shows that Hispanic students scored especially low on Constructive Use of Time, 
and although the differences were less extreme, Hispanic students also were lower than 
Black or white students on Empowerment, Commitment to Learning, and Positive 
Values, and also had low scores in the parallel School and Community contexts. Black 
students were especially strong in their Positive Identity and the parallel Personal context, 
and white students scored especially well in the Social Competencies category, and the 
parallel Social context. 


Grade Level 

The percentages of students in each grade that were in the four assets levels were not 
especially different, as seen in Table 1, with a slight tendency for gt grade students to be 
more at risk, and 12" grade students more thriving. 


Table 2, presenting the DAP sub-scale means by grade, provides a bit more perspective. 
Although these are cross-sectional data, and so trends over time cannot be inferred, 
examination of the means from 9"-12" grade shows that there was a general tendency for 
12" graders’ asset scores to be higher than 9"" graders’ scores on Commitment to 
Learning, Positive Values, Social Competencies, the Social context, and the overall DAP 
score. Smaller but still positive differences favoring 12" graders over g’ graders were 
seen for Empowerment and Positive Identity, and for the Personal, School, and 
Community contexts, as well as the Internal assets scale. 


Although cross-sectional, these data are consistent with longitudinal data from a study of 
a suburban Minneapolis community, in which it was found that assets scores declined 
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sharply over middle school (grades 6-8) on into the 9" grade, and then generally 
rebounded slightly by the 11" and especially the 1” grade (Roehlkepartain, Benson, & 
Sesma, 2003). 


Gender 


There were not large differences by gender in the proportions of students in each level of 
developmental assets, as seen in Table 1. However, Table 2 shows that females had a 
higher overall mean DAP score, with their largest advantages over males being in the 
asset categories of Commitment to Learning, and Social Competencies, and the parallel 
Social context. This finding mirrors previous research as well, since girls consistently are 
found to report more assets than boys report (Benson, Scales, & Syvertsen, 2011). 


B. Relation of Assets Level to College and Career Readiness 


Tables 3a-c and 4a-c display in two different ways the relation of assets level to the 
college and career readiness indicators. Tables 3a-3c show the percentage of students 
scoring at or above the median score for each indicator, by DAP quartiles within 
race/ethnicity, grade, and gender. Tables 4a-4c show the differences in mean scores for 
each CCR indicator, also by DAP quartiles within race/ethnicity, grade, and gender. 
Significance tests were conducted on the results in Tables 4a-4c. 


These two sets of tables show that, regardless of students’ race/ethnicity, gender, or 
grade, higher levels of developmental assets are associated with better college and career 
readiness outcomes. These associations mirror those consistently found across U.S. and 
global samples of youth and young adults, regardless of differences in the assets surveys 
used, or sample differences in race/ethnicity, age, gender, urbanicity, or socioeconomic 
composition. 


Mean CCR Differences by Race/Ethnicity 

Table 4a shows that, among Hispanic and African American students, for all eight CCR 
measures for high school students, each successive increase in DAP assets level (i.e., 
from at risk to vulnerable, vulnerable to healthy, and healthy to thriving) was associated 
with a significant increase in college and career readiness.* 


46 All of the mean differences displayed in Tables 4a-4c were significant at p < .0001. We applied a Bonferonni 
correction to the standard .05 significance level, to adjust for the fact that conducting multiple simultaneous 
significance tests can produce false significance results simply by chance. The correction (.05 level/8 
simultaneous tests per demographic category) still leaves the revised required p level at .006, a level easily 
surpassed by the results reported here. 
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White students almost showed the same complete linear trend. Among white high school 
students, each successive increase in DAP assets level was associated with a significant 
increase in six of the eight college and career readiness measures, and students in the 
highest asset level had better College Research, and Perception of Academic Rigor scores 
than white students at the other assets levels. 

In terms of the possible compensatory role of higher levels of assets, two CCR indicators 
showed either no change across race/ethnicity going from at-risk levels to thriving levels 
(Science KCS), or an expansion of inequity (white students reported greater Perception of 
Academic Rigor in their classes at the thriving level, relative to Black and Hispanic 
students, where there had been less of a difference in the overall mean, or the means at 
lower asset levels). 


However, on most of the other CCR indicators, Hispanic students may have benefitted 
the most from experiencing higher assets levels. On Math and English KCS, both school 
and family College Knowledge (support for learning about college), and, especially, 
Time Management and Study Habits, and College Research, Hispanic students at the 
highest assets level closed or erased CCR gaps with Black or white students that had 
existed overall or at lower assets levels. 


Mean CCR Differences by Grade 

Table 4b shows that, by grades, students in grades 11 and 12 generally had higher scores 
on all but two of the CCR measures (Math and Science KCS, which were essentially 
equal across grades 9-12). As seen for race/ethnicity, regardless of grade, each level of 
increase in developmental assets was associated with a parallel significant increase in 
mean CCR indicator score, for all eight indicators. Seniors in the highest assets level 
made a bigger jump in Perception of Academic Rigor than did all other students, in 
moving from the lowest (at risk) to the highest (thriving) assets level. 


At the lowest assets level, the grades were essentially equal in their Science KCS, but 
moving to the highest assets level was linked to a greater increase in Science KCS score 
for freshmen and sophomores than for juniors and seniors. For the remaining CCR 
indicators, freshmen and sophomores in the thriving assets level either closed the gap 
with juniors or seniors that existed at the at-risk assets level (English KCS, Time 
Management and Study Habits, and College Research), or eliminated that gap entirely 
(Math KCS). These results suggest that the compensatory role of assets may be greater 
for 9"" and 10" graders, who generally are at the highest risk of becoming high school 


dropouts. 


Mean CCR Differences by Gender 

Table 4c shows that, by gender, the same pattern is seen as was found for results by 
race/ethnicity and grade: For both females and males, the higher the assets level, the 
better students do on these measures of college and career readiness. In fact, females had 
a slight advantage on most of the indicators, both overall and at the lowest level of assets. 
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But on all but the College Research CCR indicator (on which there were no meaningful 
differences between the sexes), males at the highest level of assets either closed or erased 
the difference favoring females. 


Males at the “Thriving” assets level were equal to females, erasing the gap between them, 
on Math and Science KCS, and on both school and family College Knowledge (supports 
for learning about college). They closed the gender gap on English KCS, Time 
Management and Study Habits, and Perception of Academic Rigor in their core classes. 
These results suggest that increasing students’ developmental assets may play a 


particularly compensatory role for male students. 


Conclusions 
Several conclusions are apparent from these data that might be helpful to Good Shepherd 
Services and Metis Associates in using the DAP and interpreting DAP results.*” 


1. In this large sample of urban, overwhelmingly poor, primarily Hispanic and Black 
high school students, 55% were either at-risk or vulnerable, based on their assets 
scores, with Hispanic students more likely to be in those less-than-desirable assets 
levels (59% v. 46 for Black youth). 


2. So, if the future GSS samples share these demographics (as the current GSS 
samples do), having more than half those students score at the at-risk or 
vulnerable levels—especially prior to specific interventions intended to raise their 
assets—would not be surprising. Nor would it be surprising if Hispanic students 
reported a more at-risk assets profile than did Black students. 


3. On the other hand, even if this Dallas profile is more or less mirrored in GSS’s 
Brooklyn samples, that means there is still likely to be a considerable percentage 
of GSS students who, despite living in low-income or poor settings, report having 
a healthy or even a thriving assets environment. 


4. Thus, if these findings are roughly paralleled in the GSS samples, then strategies 
will be needed to maintain and build on the strengths up to half the sample may 
already be experiencing, as well as to enrich and improve on asset areas on 
which half or more of the sample may be under-nourished. 


47 Should the GSS data reveal much less association between assets levels and educational outcomes than 
reported here, then the overwhelming evidence from this and prior research, domestically and internationally, 
across diversities of cultures and samples of students, would argue that something quite unusual is then 
occurring in the GSS schools or the students’ environments. This would help GSS and Metis set more locally 
realistic expectations for improvement from that observed baseline, as well as direct them to examining 
additional data to help explain findings that ran so counter to previous research. 
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5. Focused interventions to strengthen students’ experience of the weaker asset areas 
could result in a meaningful improvement in the overall assets profile of GSS 
students. Consider that 37% of Black students and 44% of Hispanic students in 
the Dallas sample scored in the “vulnerable” level. If GSS interventions can 
achieve an average change from “somewhat or sometimes” to “very or often” in 
how much or how often students feel or experience those assets, this would lift 
the average “vulnerable” student into the “healthy” level. 


6. Achieving that kind of impact (students going from “vulnerable” to “healthy”’) is 
likely to have a substantial effect on students’ well-being, academic and 
otherwise. Numerous studies have found that every increase in assets level brings 
with it significant improvements in key youth outcomes, from better school 
attendance and better grades, to greater volunteering and other positive behaviors, 
and lessened engagement in substance use, violence, and a host of other high-risk 
behaviors (see the references cited in this report). Practically every psychological, 
socio-emotional, and behavioral outcome thus improved by students’ 
experiencing higher assets levels also has an indirect association with promoting 
better orientation to school, greater effort, and stronger academic performance. 


7. On top of these well-documented general effects of higher assets levels on overall 
youth well-being, the Dallas data specifically show that, for virtually every 
demographic group, on every one of the college and career outcomes studied, 
every increase in assets level experienced was associated with a statistically 
significant increase in that indicator of college and career readiness. GSS students 
similar to this sample demographically would be expected to show a similar 
positive correlation of assets with educational outcomes, thus making an increase 
in assets level over time a key supplemental indicator of possible GSS 
effectiveness in promoting greater academic success for its students. 


8. Although speculative without further analysis, these results also suggest that 
student experience of developmental assets may be especially valuable to 
Hispanic, male, and 9" and 10" grade students; In general, students from those 
groups who were at the “thriving” assets level had college and career readiness 
means that were closer or equal to those of females, Blacks and whites, and 
juniors and seniors, eliminating or erasing gaps that had existed overall or at 
lower assets levels among those groups. This is especially important because even 
in this poor urban population, some students are still more vulnerable than others, 
and these results suggest that increasing students developmental assets might be 
the most helpful to these more vulnerable students. 
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A Final Note 

The more assets these students reported, the more they used problem-solving and critical 
thinking skills in their classroom work in Math, English, and Science, the more they were 
required to hypothesize and explain, to present their work to others, and the more 
academically challenging they found their coursework to be. In addition, the higher their 
level of assets, the more they aspired to go to college, the more research they undertook 
on colleges, and the more support they got from family and school in learning about 
colleges, admission requirements, and financial aid. These results were obtained, not in 
an affluent, suburban, white population, but in a poor, urban, Hispanic and African 
American population. Thus, it is reasonable for GSS to anticipate finding similar relations 
between assets levels and key educational outcomes in their work with Brooklyn students 
who have a similar demographic profile to these Dallas students. 


Clearly, there is a correlation between higher assets levels and improved student college 
and career readiness indicators. But the mechanism of this relationship may have both 
direct and indirect components. Directly, in terms of effects on students, higher assets 
levels reflect more student engagement and more support from others, at the least. 


But indirectly, higher student assets levels may affect teachers as well. Since this was just 
a cross-sectional and not a longitudinal study, it could well be that students who are 
challenged and supported to use higher-order thinking skills, and who experience their 
teachers expecting more of them, report more assets as a result. But given that 
developmental cause-effect relations are typically bi-directional (Benson, Scales, 
Hamilton, & Sesma, 2006), it is at least as likely that students who first have higher 
levels of assets also produce an effect on their teachers, encouraging their teachers to feel 
more capable and motivated, and therefore to become better, more effective educators. 

AS we write in a paper in progress on these data (Scales, Pekel, & Roehlkepartain, 2012), 


“Teachers might become better teachers (as reflected in students having higher scores on 
the key cognitive skills being experienced in their classrooms) in part because their 
students have the developmental supports that make them good partners in the teaching 
and learning collaboration. Beyond the effect that individual students’ or a classroom’s 
level of assets may have, there may be structural effects of a school community that has a 
commitment to building student assets. For example, a recent study found that, in schools 
that were characterized by a stronger culture of building students’ assets, teachers in turn 
felt more motivated as educators and had greater job satisfaction, both of which are 
linked to better pedagogical performance (Butler, 2010). It has been noted that schools 
that are “great places to learn” also are great places to teach (Starkman, Scales, & 
Roberts, 2006), and these results would appear to support that contention.” 


The combined strength of these associations found in a sample very similar to the GSS 


samples demographically, and the substantial literature finding similar associations 
between assets and positive outcomes across wide diversities of demographics and 
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outcomes, suggests that GSS students are highly likely also to show a significant linkage 
between their assets levels and a variety of school success outcomes. This suggests that 
the Developmental Assets Profile can be a compelling data source for both stimulating 
and documenting change in students’ strengths and outcomes, and therefore a valuable 
resource for helping to demonstrate the contribution of GSS programming to 
improvement of students’ educational success. 
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Figure 1 


Developmental Asset Profile (DAP) Mean Scale 
Scores by Race/Ethnicity 
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@ HISPANIC 
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Figure 2 


Time Management and Study Habits Mean Scale 
Scores by Level of Developmental Assets and 
Grade 
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Figure 3 


College Knowledge - School Supports Mean Scale 
Scores, by Level of Developmental Assets and 
Gender 
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m@ MALE 
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Table 1 


Percentage of Dallas ISD High School Students in Developmental Assets Levels, Total, 
and by Race/Ethnicity, Gender, and Grade“® 


At-Risk Vulnerable Healthy Thriving 
Dallas ISD Overall 13 42 34 11 
U.S. Field Test Study 16 40 34 10 
(2005) (HS only, N=706) 
Race/Ethnicity 
Black (N=4773) 9 37 40 15 
Hispanic N=13,361) 15 44 32 9 
White (N=1,120) 9 36 38 16 
Gender 
Female (N=10,593) 12 41 36 11 
Male (N=9,524) 15 43 32 10 
Grade 
9 (N=6,323) 15 42 34 10 
10" (N=5,778) 14 42 34 10 
11" (N=4,469) 12 43 35 11 
12" (N=3,547) 12 40 35 12 


Total N=20,117 


48 To keep the tables as simple as possible, sub-ce 
overall sample size meant that the smallest sub-ce 


sample sizes are not reported here. However, the large 


size in all these analyses was n 2 135, with the smallest sub- 


cell sizes in any one table typically several times larger than that. Thus, the percentages and means reported 
here do not have the substantial error that would be the case if they were based on small sub-cells of < 100 or 


even fewer students. 
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Dallas High School Students’ Mean Developmental Assets Profile Scale Scores, by 
Gender, Race/Ethnicity, and Grade 


Overall 
Mean Gender Race/Ethnicity Grade 

Female | Male Black | Hispanic | White 9 10 11 12 
Support 20.53 20.82 20.21 21.38 | 20.19 21.06 20.69 | 20.46 | 20.39 | 20.51 
Empower- 20.44 20.80 20.05 21.28 | 20.03 21.88 20.26 | 20.30 | 20.54 | 20.89 
ment 
Boundaries & | 20.42 20.77 20.03 20.89 | 20.20 20.82 20.63 | 20.36 | 20.28 | 20.31 
Expectations 
Constructive 16.86 16.85 16.87 19.58 | 15.71 18.92 16.89 | 16.78 | 16.93 | 16.83 
Use of Time 
Commitment 20.43 21.16 19.62 21.67 | 19.84 21.68 19.80 | 20.21 | 21.03 | 21.14 
to Learning 
Positive 20.51 21.00 19.96 21.27 | 20.10 21.85 19.99 | 20.37 | 20.91 | 21.17 
Values 
Social 20.72 21.40 19.95 20.77 | 20.53 22.37 20.20 | 20.54 | 21.11 | 21.42 
Competencies 
Positive 21.61 21.41 21.83 23.21 | 21.10 21.39 21.24 | 21.54 | 21.91 | 22.02 
Identity 
Personal 21.24 21.43 21.03 22.50 | 20.77 21.64 20.84 | 21.18 | 21.54 | 21.67 
Social 20.91 21.54 20.22 21.36 | 20.62 22.35 20.49 | 20.71 | 21.26 | 21.55 
Family 22.67 23.08 22.21 23.40 | 22.43 22.67 22.64 | 22.75 | 22.69 | 22.55 
School 19.71 20.08 19.31 20.32 | 19.36 20.68 19.57 | 19.50 | 19.88 | 20.10 
Community 17.48 17.79 17.14 18.95 | 16.77 19.64 17.31 | 17.28 | 17.65 | 17.92 
External 19.56 19.81 19.29 20.78 | 19.04 20.67 19.62 | 19.48 | 19.54 | 19.64 
Internal 20.82 21.24 20.34 21.73 | 20.39 21.82 20.31 | 20.67 | 21.24 | 21.44 
Total DAP 40.39 41.05 39.63 42.51 | 39.43 42.49 39.92 | 40.14 | 40.78 | 41.07 
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Table 3a 
Differences in Percentage of Dallas High School Students At or Above Median Mean 


College and Career Readiness (CCR) Scale Score, by Level of Developmental Assets 
(Total DAP Score), Total, and by Race/Ethnicity 


Asset Levels 


(N=20,241) At-Risk Vulnerable Healthy Thriving 
Math KCS* 28 41 57 71 
Black 27 42 59 72 
Hispanic 28 42 59 72 
White 28 30 45 63 
English/Language Arts KCS 24 41 59 74 
Black 25 40 61 73 
Hispanic 24 41 58 73 
White 24 49 65 79 
Science KCS 27 41 57 70 
Black 31 42 57 69 
Hispanic 27 41 57 71 
White 16 43 58 68 
Time Management & Study Habits 17 40 67 82 
Black 21 42 68 79 
Hispanic 16 39 66 82 
White 18 43 73 91 
Perception of Academic Rigor 46 52 59 64 
Black 38 46 55 60 
Hispanic 48 54 60 65 
White 42 51 57 75 
College Research 39 47 55 65 
Black 49 57 63 70 
Hispanic 36 43 50 62 
White 35 51 57 58 
College Knowledge School Supports 25 38 52 65 
Black 30 42 55 69 
Hispanic 25 37 52 65 
White 14 33 47 60 
College Knowledge Family Supports 25 39 55 68 
Black 29 49 63 74 
Hispanic 25 37 52 65 
White 21 44 58 65 


*KCS=Key Cognitive Strategies 
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Table 3b 
Differences in Percentage of Dallas High School Students At or Above Median Mean 


College and Career Readiness (CCR) Scale Score, by Level of Developmental Assets 
(Total DAP Score), Total, and by Grade 


Asset Levels 


(N=20,241) At-Risk Vulnerable Healthy Thriving 
Math KCS* 28 41 57 71 
9" Grade 26 A1 58 72 
10" Grade 26 A1 59 72 
11" Grade 32 44 58 71 
12" Grade 29 40 57 69 
English/Language Arts KCS 24 41 59 74 
9" Grade 19 35 56 68 
10" Grade 23 A1 58 75 
11" Grade 33 47 65 77 
12" Grade 27 44 62 76 
Science KCS 27 41 57 70 
9" Grade 28 42 61 74 
10" Grade 26 A1 58 71 
11" Grade 27 40 51 64 
12" Grade 29 A1 56 70 
Time Management & Study Habits 17 40 67 82 
9" Grade 13 34 64 81 
10" Grade 16 40 66 81 
11" Grade 23 AG 71 84 
12" Grade 18 44 70 82 
Perception of Academic Rigor 46 52 59 64 
9 Grade 45 49 65 62 
10" Grade 47 53 59 64 
11" Grade 51 57 64 66 
12" Grade 43 52 59 66 
College Research 39 47 55 65 
9" Grade 29 34 40 48 
10" Grade 33 40 49 59 
11" Grade 44 55 62 74 
12" Grade 59 70 77 85 


(Continued on next page) 
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Table 3b (cont) 


(N=20,241) At-Risk Vulnerable Healthy Thriving 
College Knowledge School 25 38 52 65 
Supports 
9" Grade 21 28 42 56 
10" Grade 21 31 46 60 
11" Grade 28 43 56 68 
12" Grade 41 61 76 84 
College Knowledge Family Supports 25 39 55 68 
9" Grade 22 33 46 57 
10" Grade 25 36 51 65 
11" Grade 28 Al 59 70 
12" Grade 32 56 74 85 


*KCS=Key Cognitive Strategies 
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Table 3c 


Differences in Percentage of Dallas High School Students At or Above Median Mean 
College and Career Readiness (CCR) Scale Score, by Level of Developmental Assets 


(Total DAP Score), Total, and by Gender 


Asset Levels 


(N=20,241) At-Risk Vulnerable Healthy Thriving 
Math KCS* 28 41 57 71 
Female 31 43 58 69 
Male 25 40 58 74 
English/Language Arts KCS 24 41 59 74 
Female 29 47 64 78 
Male 20 35 55 68 
Science KCS 27 41 57 70 
Female 31 41 57 70 
Male 25 41 57 71 
Time Management & Study Habits 17 40 67 82 
Female 21 46 72 86 
Male 13 33 60 77 
Perception of Academic Rigor 46 52 59 64 
Female 50 54 60 64 
Male 43 51 58 64 
College Research 39 47 55 65 
Female 37 46 54 64 
Male 39 48 55 66 
College Knowledge School Supports 25 38 52 65 
Female 27 38 52 65 
Male 24 38 54 67 
College Knowledge Family Supports 25 39 55 68 
Female 25 38 55 68 
Male 26 41 56 68 


*KCS=Key Cognitive Strategies 
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Table 4a 

Differences in Mean College and Career Readiness (CCR) Scale Scores,* by Level of 
Developmental Assets (Total DAP Score) and Race/Ethnicity, Dallas High School 
Students** 


Overall 
F Mean At-Risk | Vulnerable | Healthy | Thriving 
Math KCS*** Black (3,4440)=159.99 28.33 24.17d | 26.71c 29.41b 31.75a 
Hispanic (3,12966)=497.27 27.14 23.95d | 26.66c 29.12b 31.58a 
White (3,1079)=33.49 26.38 23.11d | 25.00c 26.98b 29.33a 
English KCS Black (3,4473)=190.34 31.29 26.14d | 29.56c 32.58b 34.96a 
Hispanic (3,12922)=1301.80 | 30.19 25.52d | 29.45c 32.22b 34.85a 
White (3,1093)=61.17 31.66 25.34d | 30.28c 32.86b 35.45a 
Science KCS Black (3,4442)=111.52 28.84 24.80d | 27.35c 29.81b | 32.11a 
Hispanic (3,12773)=392.34 28.17 24.52d | 27.38c 29.93b 32.07a 
White (3,1090)=41.09 28.63 23.32d | 27.12c 29.88b 32.03a 
Time Mgt & Black (3,4331)=412.56 30.40 25.04d | 28.55c 31.83b 34.19a 
Study Habits Hispanic (3,12701)=1517.75 | 28.99 23.64d | 28.13c 31.35b | 34.13a 
White (3,1068)=150.28 30.44 23.50d | 28.53c 31.97b 35.11a 
College Black (3,4210)=35.44 20.45 22.09d | 21.05c 20.03b 19.27a 
Research**** | Hispanic (3,12085)=146.16 22.27 29.94d | 22.65c 21.55b | 20.14a 
White (3,1044)=17.67 20.99 24.39d | 21.21b 20.27a 20.34a 
Perception of | Black (3,4533)=37.92 28.70 26.75d | 27.84c 29.25b | 30.48a 
Academic Hispanic (3,12969)=82.13 29.15 27.57d | 28.88c 29.87b | 30.67a 
Rigor White (3,1089)=21.55 29.18 27.30¢ | 28.02b,c 29.51b | 32.11a 
College Black (3,4019)=156.32 28.05 23.37d | 26.15c 28.99b 32.69a 
Knowledge - Hispanic (3,11671)=460.35 25.99 21.83d | 24.94c 28.02b | 31.00a 
School White (3,1015)=42.44 25.81 20.01d | 24.56c 26.78b 29.81a 
Supports 
College Black (3,4203)=232.75 28.13 21.35d | 25.85c 29.57b | 33.65a 
Knowledge - Hispanic (3,11994)=507.82 24.48 19.83d | 23.18c 26.88b | 30.26a 
Family White (3,1026)=50.87 26.36 19.30d | 24.47c 28.01b 30.78a 
Supports 


** Within the same row of means by DAP quartile, means with differing superscripts (a,b,c,d) are statistically different from each 
other. Yellow-highlighted superscripts indicate where significant differences in means did not follow a strict linear correlation with 
rise in assets level. 

**”Other” racial/ethnic category not shown, since multiple races/ethnicities are represented, and the results are therefore not clearly 
interpretable 

***K CS=Key Cognitive Strategies 

**T ower score indicates students were more involved in researching colleges. 
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Table 4b 


Differences in Mean College and Career Readiness (CCR) Scale Scores,* by Level of 
Developmental Assets (Total DAP Score) and Grade, Dallas High School Students 


Overall 
F Mean | At-Risk | Vulnerable | Healthy | Thriving 
Math Kcs** | 9" (3,5953)=264.06 | 27.46 | 23.50d | 26.57c 29.13b | 31.53a 
10™ | (3,5468)=235.04 | 27.63 | 23.77d | 26.69c 29.24b | 31.61a 
11™ | (3,4267)=128.83 | 27.94 | 24.68d | 26.98c 29.13b | 31.42a 
12™ | (3,3309)=98.69 27.42 | 24.35d | 26.04c 28.66b | 31.29a 
English KCS g* (3,5941)=307.84 | 29.57 24.50d | 28.51c 31.63b | 33.99a 
10" | (3,5434)=276.55 | 30.38 25.43d | 29.41c 32.21b | 35.01a 
11™ | (3,4285)=168.20 | 31.56 | 27.18d | 30.46c 33.09b | 35.66a 
12™ | (3,3372)=146.70 | 31.48 26.60d | 30.44c 32.95b | 35.37a 
Science KCS g” (3,5850)=229.79 | 28.77 | 24.80d | 27.72c 30.61b | 32.79a 
10™ | (3,5404)=186.60 | 28.34 | 24.29d | 27.36c 29.99b | 32.27a 
11™ | (3,4238)=88.64 27.96 | 24.60d | 27.02c 29.21b | 31.20a 
12™ | (3,3360)=85.58 28.24 | 24.31d | 27.16c 29.52b | 31.80a 
Time Mgt & 9” (3,5811)=773.32 | 28.72 | 22.97d | 27.57c 31.09b | 34.04a 
Study Habits | 10™ | (3,5338)=644.43 | 29.41 | 23.86d | 28.37c 31.51b | 33.33a 
11™ | (3,4187)=432.59 | 30.13. | 25.11d | 28.88c 31.98b | 34.57a 
12™ | (3,3297)=374.52 | 30.07. | 25.47d | 28.70c 31.99b | 34.29a 
College 9” (3,5561)=65.04 23.60 | 24.94d | 24.15c 22.91b | 21.69a 
Research*** | 10" (3,5104)=80.27 22.45 24.48d | 22.95c 21.59b | 20.59a 
11™ | (3,4013)=54.50 20.53 | 22.49d | 21.00c 19.89b | 19.69a 
12™ | (3,3182)=29.22 18.87 | 20.49d | 19.14c 18.46b | 17.72a 
Perception of | 9" (3,5975)=31.42 28.63 27.33d | 28.21c 29.27b | 30.12a 
Academic 10" | (3,5530)=37.54 29.03 27.31d | 28.72c 29.68b | 30.54a 
Rigor 11™ | (3,4298)=25.79 29.63 28.30¢ 29.14c 30.52b | 31.40a 
12™ | (3,3341)=35.50 29.21 27.09d | 28.59c 29.89b | 31.38a 
College 9” (3,5324)=192.34 | 24.43 | 20.70d | 23.26c 26.13b | 29.16a 
Knowledge- 10™ | (3,4917)=206.18 | 25.17 | 20.96d | 24.02c 26.79b | 30.24a 
School 11™ | (3,3859)=138.72 | 27.28 23.17d | 25.97c 28.88b | 31.57a 
Supports 12™ | (3,3105)=128.55 | 31.11 25.47d | 29.72 32.81b | 35.60a 
College 9” (3,5481)=207.85 | 24.02 19.50d | 22.79c 26.01b | 29.06a 
Knowledge— | 10" (3,5063)=221.76 | 24.65 19.88d | 23.13c 26.80b | 30.14a 
Family 11" | (3,3978)=207.43 | 25.76 | 20.47d | 23.75c 28.05b | 31.73a 
Supports 12" (3,3218)=253.11 | 28.96 21.25d | 26.85c 31.44b | 35.56a 


* Within the same row of means by DAP quartile, means with differing superscripts (a,b,c,d) are statistically different from each 
indicate where significant differences in means did not follow a strict linear correlation with 


other. 
rise in assets level. 


**KCS=Key Cognitive Strategies 


**T ower score indicates students were more involved in researching colleges 
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Table 4c 


Differences in Mean College and Career Readiness (CCR) Scale Scores,* by Level of 
Developmental Assets (Total DAP Score) and Gender, Dallas High School Students 


Overall 
F Mean | At-Risk | Vulnerable | Healthy | Thriving 

Math KCS** | Female | (3,9994)=291.52 | 27.90 | 24.70d | 26.84c 29.05b | 31.29a 

Male (3,9005)=419.50 | 27.29 | 23.32d | 26.36c 29.12b | 31.72a 
English KCS Female | (3,10060)=395.19 | 31.56 | 27.13d | 30.49c 32.89b | 35.57a 

Male (3,8974)=482.55 29.45 24.40d | 28.55c 31.71b | 34.10a 
Science KCS Female | (3,9970)=248.70 | 28.67 | 25.16d | 27.49c 29.98b | 32.11a 

Male (3,8884)=318.97 | 28.04 | 23.99d | 27.22c 29.83b | 32.04a 
Time Met & Female | (3,34)=172.34 30.46 | 25.12d | 29.13c 32.23b | 34.82a 
Study Habits | Male (3,881)=1032.43 | 28.37 | 22.88d | 27.41c 30.71b | 33.59a 
College Female | (3,9521)=127.63 | 21.63 | 23.56d | 22.21c 20.97b | 19.85a 
Research*** | Male (3,8341)=106.89 | 21.86 | 23.69d | 22.24c 21.08b | 19.90a 
Perception Female | (3,10092)=145.42 | 29.43 | 28.29d | 28.99c 29.80b | 30.94a 
of Academic | Male (3,9054)=82.33 28.67 | 26.79d | 28.25c 29.56b | 30.59a 
Rigor 
College Female | (3,9202)=325.97 | 26.60 | 22.31d | 25.14c 28.04b | 31.25a 
Knowledge- | Male (3,8005)=382.41 | 26.36 | 21.75d | 25.28c 28.44b | 31.58a 
School 
Supports 
College Female | (3,9461)=446.56 | 25.59 | 20.08d | 23.67c 27.59b | 31.24a 
Knowledge— | Male (3,8281)=435.17 | 25.37 | 20.07d | 23.97c 27.90b | 31.51a 
Family 
Supports 


* Within the same row of means by DAP quartile, means with differing superscripts (a,b,c,d) are statistically different from each 
other. indicate where significant differences in means did not follow a strict linear correlation with 
rise in assets level. 

**KCS=Key Cognitive Strategies 

**T ower score indicates students were more involved in researching colleges. 
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